Debian Bug report logs - #735169
os-prober cannot work with the latest upstream blkid (util-linux-2.24)

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: kcao <cc.5nth@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: os-prober cannot work with the latest upstream blkid (util-linux-2.24)

Date: Mon, 13 Jan 2014 12:20:23 -0800

Package: os-prober
Version: 1.63
Severity: wishlist
Tags: upstream

Dear Maintainer,

The new blkid in util-linux-2.24 is able to recognize the extend
partition, but os-prober relies on the return value "2" (exit_not_found)
of blkid to skip processing the extend partition.

Then, with the new version of util-linux, os-prober will try
to mount the extend partition with FS types found in the system,
which is dangerous.

e.g.,

# fdisk -l
....
Device    Boot     Start       End    Blocks  Id System
/dev/sda1 *           63 252847034 126423486   7 HPFS/NTFS/exFAT
/dev/sda2      252847035 625137344 186145155   5 Extended
/dev/sda5      252847098 253425374    289138+ 83 Linux
/dev/sda6      385254828 482913899  48829536  83 Linux
....


blkid 2.24-2
---
# blkid /dev/sda2
/dev/sda2: PTTYPE="dos" PARTUUID="28b48ca1-02"
# echo $?
0

# blkid -o value -s TYPE /dev/sda2  # no output
# echo $?
0


blkid 2.20.0
---
# blkid /dev/sda2  # no output
# echo $?
2
# blkid -o value -s TYPE /dev/sda2  # no output
# echo $?
2


Affected os-prober code:
---
File: /usr/lib/os-probes/50mounted-tests
-
8   types="$(fs_type "$partition")" || types=NOT-DETECTED
9   if [ "$types" = NOT-DETECTED ]; then
10      debug "$1 type not recognised; skipping"
11      exit 0


With the old blkid (util-linux-2.20),
os-prober will just skip here when handling /dev/sda2,
but with the new version of blkid,
os-prober will go on and try to mount /dev/sda2 with
FS types found in the system, which causes lots of errors,
even kernel calltrace when trying xfs.


This is not a problem now because util-linux-2.20 is currently
used, but when someday util-linux-2.24 is introduced in,
this would be a serious problem.


-- System Information:
Debian Release: jessie/sid
  APT prefers testing-updates
  APT policy: (500, 'testing-updates'), (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 3.12-1-amd64 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages os-prober depends on:
ii  libc6  2.17-97

os-prober recommends no packages.

os-prober suggests no packages.

-- no debconf information

Message #10 received at 735169@bugs.debian.org (full text, mbox, reply):

From: Andreas Henriksson <andreas@fatal.se>

To: 735169@bugs.debian.org

Cc: kcao <cc.5nth@gmail.com>, xnox@debian.org

Subject: os-prober cannot work with the latest upstream blkid (util-linux-2.24)

Date: Sat, 12 Jul 2014 00:55:08 +0200

[Message part 1 (text/plain, inline)]

Control: tags -1 + patch

Hello!

I'm working on an updated util-linux package (v2.24.2) and was pointed
to this os-prober bug report.

In my limited testing I haven't been able to reproduce any problems
here with os-prober and util-linux 2.24.2.

I can confirm though that the behaviour of running blkid against
an extended dos partition has changed. In the new version information
is correctly reported for those instead of just erroring out.

The old blkid version (2.20.1) returns error code 2 when running
against an extended dos partition.

$ sudo blkid -V
blkid from util-linux 2.24.2  (libblkid 2.24.0, 24-Apr-2014)

$ sudo blkid -o export /dev/sda2
DEVNAME=/dev/sda2
PTUUID=dd6e951b
PTTYPE=dos
PARTUUID=000ae69f-02

$ sudo blkid /dev/sda2
/dev/sda2: PTUUID="dd6e951b" PTTYPE="dos" PARTUUID="000ae69f-02"


I'm attaching a patch that should hopefully help. It tries to make the
executed codepath similar to what it was by adding an extra check on the
case where value of TYPE is "" and then exits 0 if it looks like an
extended dos partition.

Any feedback would be greatly appreciated.

Regards,
Andreas Henriksson

[os-prober-util-linux-2.24.patch (text/x-diff, attachment)]

Message #17 received at 735169@bugs.debian.org (full text, mbox, reply):

From: kcao <cc.5nth@gmail.com>

To: 735169@bugs.debian.org

Cc: Andreas Henriksson <andreas@fatal.se>, xnox@debian.org

Subject: Re: os-prober cannot work with the latest upstream blkid (util-linux-2.24)

Date: Wed, 8 Oct 2014 08:12:48 +0800

Hi,

This patch works good for me.

-kcao

On Sat, Jul 12, 2014 at 12:55:08AM +0200, Andreas Henriksson wrote:
> Control: tags -1 + patch
> 
> Hello!
> 
> I'm working on an updated util-linux package (v2.24.2) and was pointed
> to this os-prober bug report.
> 
> In my limited testing I haven't been able to reproduce any problems
> here with os-prober and util-linux 2.24.2.
> 
> I can confirm though that the behaviour of running blkid against
> an extended dos partition has changed. In the new version information
> is correctly reported for those instead of just erroring out.
> 
> The old blkid version (2.20.1) returns error code 2 when running
> against an extended dos partition.
> 
> $ sudo blkid -V
> blkid from util-linux 2.24.2  (libblkid 2.24.0, 24-Apr-2014)
> 
> $ sudo blkid -o export /dev/sda2
> DEVNAME=/dev/sda2
> PTUUID=dd6e951b
> PTTYPE=dos
> PARTUUID=000ae69f-02
> 
> $ sudo blkid /dev/sda2
> /dev/sda2: PTUUID="dd6e951b" PTTYPE="dos" PARTUUID="000ae69f-02"
> 
> 
> I'm attaching a patch that should hopefully help. It tries to make the
> executed codepath similar to what it was by adding an extra check on the
> case where value of TYPE is "" and then exits 0 if it looks like an
> extended dos partition.
> 
> Any feedback would be greatly appreciated.
> 
> Regards,
> Andreas Henriksson

> diff --git a/common.sh b/common.sh
> index 30e245e..8734b62 100644
> --- a/common.sh
> +++ b/common.sh
> @@ -116,6 +116,29 @@ fs_type () {
>  	fi
>  }
>  
> +is_dos_extended_partition() {
> +	if type blkid >/dev/null 2>&1; then
> +		local output
> +
> +		output="$(blkid -o export $1)"
> +
> +		# old blkid (util-linux << 2.24) errors out on extended p.
> +		if [ "$?" = "2" ]; then
> +			return 0
> +		fi
> +
> +		# dos partition type and no filesystem type?...
> +		if echo $output | grep -q ' PTTYPE=dos ' &&
> +				! echo $output | grep -q ' TYPE='; then
> +			return 0
> +		else
> +			return 1
> +		fi
> +	fi
> +
> +	return 1
> +}
> +
>  parse_proc_mounts () {
>  	while read -r line; do
>  		set -f
> diff --git a/linux-boot-probes/common/50mounted-tests b/linux-boot-probes/common/50mounted-tests
> index 41e26e6..91b6dd9 100755
> --- a/linux-boot-probes/common/50mounted-tests
> +++ b/linux-boot-probes/common/50mounted-tests
> @@ -20,6 +20,10 @@ elif [ "$types" = ntfs ]; then
>  		types='ntfs-3g ntfs'
>  	fi
>  elif [ -z "$types" ]; then
> +	if is_dos_extended_partition "$partition"; then
> +		debug "$1 looks like an extended dos partition; skipping"
> +		exit 0
> +	fi
>  	if type cryptsetup >/dev/null 2>&1 && \
>  	   cryptsetup luksDump "$partition" >/dev/null 2>&1; then
>  		debug "$1 is a LUKS partition; skipping"

Message #22 received at 735169@bugs.debian.org (full text, mbox, reply):

From: Andrii Senkovych <jolly_roger@itblog.org.ua>

To: 735169@bugs.debian.org

Subject: os-prober cannot work with the latest upstream blkid (util-linux-2.24)

Date: Thu, 23 Oct 2014 18:14:17 +0300

Hello,

I've tested the patch as well and it works. Thank you, Andreas!

-- 
Best regards, Andrii Senkovych

Message #27 received at 735169@bugs.debian.org (full text, mbox, reply):

From: Paweł Tomulik <ptomulik@meil.pw.edu.pl>

To: 735169@bugs.debian.org

Subject: Re: os-prober cannot work with the latest upstream blkid (util-linux-2.24)

Date: Thu, 8 Oct 2015 01:51:01 +0200

On Thu, 23 Oct 2014 18:14:17 +0300 Andrii Senkovych
<jolly_roger@itblog.org.ua> wrote:
> Hello,
> 
> I've tested the patch as well and it works. Thank you, Andreas!
> 
> -- 
> Best regards, Andrii Senkovych
> 
> 


Seems I have this problem:


ptomulik@barakus:$ LANG=C dpkg -l util-linux os-prober
Desired=Unknown/Install/Remove/Purge/Hold
|
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                         Version             Architecture
 Description
+++-============================-===================-===================-==============================================================
ii  os-prober                    1.67                amd64
 utility to detect other OSes on a set of drives
ii  util-linux                   2.27-3              amd64
 Miscellaneous system utilities


Every kernel/grub update floods my logs with these entries (sda4 and
sdb4 are extended partitions):

paź 08 01:22:45 barakus os-prober[18910]: debug: /dev/sda1: part of
software raid array
paź 08 01:22:45 barakus os-prober[18916]: debug: running
/usr/lib/os-probes/50mounted-tests on /dev/sda4
paź 08 01:22:45 barakus kernel: EXT4-fs (sda4): unable to read superblock
paź 08 01:22:45 barakus kernel: EXT4-fs (sda4): unable to read superblock
paź 08 01:22:45 barakus kernel: EXT4-fs (sda4): unable to read superblock
paź 08 01:22:45 barakus kernel: XFS (sda4): Invalid superblock magic number
paź 08 01:22:45 barakus kernel: FAT-fs (sda4): utf8 is not a recommended
IO charset for FAT filesystems, filesystem will be case sensitive!
paź 08 01:22:45 barakus kernel: FAT-fs (sda4): bogus number of reserved
sectors
paź 08 01:22:45 barakus kernel: FAT-fs (sda4): Can't find a valid FAT
filesystem
paź 08 01:22:45 barakus kernel: FAT-fs (sda4): utf8 is not a recommended
IO charset for FAT filesystems, filesystem will be case sensitive!
paź 08 01:22:45 barakus kernel: FAT-fs (sda4): bogus number of reserved
sectors
paź 08 01:22:45 barakus kernel: FAT-fs (sda4): Can't find a valid FAT
filesystem
paź 08 01:22:45 barakus kernel: ntfs: (device sda4):
read_ntfs_boot_sector(): Primary boot sector is invalid.
paź 08 01:22:45 barakus kernel: ntfs: (device sda4):
read_ntfs_boot_sector(): Mount option errors=recover not used. Aborting
without trying to recover.
paź 08 01:22:45 barakus kernel: ntfs: (device sda4): ntfs_fill_super():
Not an NTFS volume.
paź 08 01:22:45 barakus kernel: MINIX-fs: unable to read superblock
paź 08 01:22:45 barakus kernel: attempt to access beyond end of device
paź 08 01:22:45 barakus kernel: sda4: rw=16, want=3, limit=2
paź 08 01:22:45 barakus kernel: hfsplus: unable to find HFS+ superblock
paź 08 01:22:45 barakus kernel: qnx4: no qnx4 filesystem (no root dir).
paź 08 01:22:45 barakus kernel: ufs: You didn't specify the type of your
ufs filesystem

                                 mount -t ufs -o
ufstype=sun|sunx86|44bsd|ufs2|5xbsd|old|hp|nextstep|nextstep-cd|openstep ...

                                 >>>WARNING<<< Wrong ufstype may corrupt
your filesystem, default is ufstype=old
paź 08 01:22:45 barakus kernel: hfs: can't find a HFS filesystem on dev sda4
paź 08 01:22:45 barakus os-prober[18951]: debug: /dev/sda5: is active swap
paź 08 01:22:45 barakus os-prober[18955]: debug: /dev/sda6: part of
software raid array
paź 08 01:22:45 barakus os-prober[18959]: debug: /dev/sda7: part of
software raid array
paź 08 01:22:45 barakus os-prober[18963]: debug: /dev/sdb1: part of
software raid array
paź 08 01:22:45 barakus os-prober[18969]: debug: running
/usr/lib/os-probes/50mounted-tests on /dev/sdb4
paź 08 01:22:45 barakus kernel: EXT4-fs (sdb4): unable to read superblock
paź 08 01:22:45 barakus kernel: EXT4-fs (sdb4): unable to read superblock
paź 08 01:22:45 barakus kernel: EXT4-fs (sdb4): unable to read superblock
paź 08 01:22:45 barakus kernel: XFS (sdb4): Invalid superblock magic number
paź 08 01:22:45 barakus kernel: FAT-fs (sdb4): utf8 is not a recommended
IO charset for FAT filesystems, filesystem will be case sensitive!
paź 08 01:22:45 barakus kernel: FAT-fs (sdb4): bogus number of reserved
sectors
paź 08 01:22:45 barakus kernel: FAT-fs (sdb4): Can't find a valid FAT
filesystem
paź 08 01:22:45 barakus kernel: FAT-fs (sdb4): utf8 is not a recommended
IO charset for FAT filesystems, filesystem will be case sensitive!
paź 08 01:22:45 barakus kernel: FAT-fs (sdb4): bogus number of reserved
sectors
paź 08 01:22:45 barakus kernel: FAT-fs (sdb4): Can't find a valid FAT
filesystem
paź 08 01:22:45 barakus kernel: ntfs: (device sdb4):
read_ntfs_boot_sector(): Primary boot sector is invalid.
paź 08 01:22:45 barakus kernel: ntfs: (device sdb4):
read_ntfs_boot_sector(): Mount option errors=recover not used. Aborting
without trying to recover.
paź 08 01:22:45 barakus kernel: ntfs: (device sdb4): ntfs_fill_super():
Not an NTFS volume.
paź 08 01:22:45 barakus kernel: MINIX-fs: unable to read superblock
paź 08 01:22:45 barakus kernel: attempt to access beyond end of device
paź 08 01:22:45 barakus kernel: sdb4: rw=16, want=3, limit=2
paź 08 01:22:45 barakus kernel: hfsplus: unable to find HFS+ superblock
paź 08 01:22:45 barakus kernel: qnx4: no qnx4 filesystem (no root dir).
paź 08 01:22:45 barakus kernel: ufs: You didn't specify the type of your
ufs filesystem

                                 mount -t ufs -o
ufstype=sun|sunx86|44bsd|ufs2|5xbsd|old|hp|nextstep|nextstep-cd|openstep ...

                                 >>>WARNING<<< Wrong ufstype may corrupt
your filesystem, default is ufstype=old
paź 08 01:22:45 barakus kernel: hfs: can't find a HFS filesystem on dev sdb4



Disk layout:

ptomulik@barakus:$ /sbin/fdisk -l /dev/sda
Disk /dev/sda: 465,8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5f572a40

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sda1            2048   1050623   1048576   512M fd Linux raid
autodetect
/dev/sda4         1050624 976773167 975722544 465,3G  5 Extended
/dev/sda5         1052672  17829887  16777216     8G 82 Linux swap / Solaris
/dev/sda6        17831936 152049663 134217728    64G fd Linux raid
autodetect
/dev/sda7       152051712 976773167 824721456 393,3G fd Linux raid
autodetect



-- 
Pawel Tomulik

Message #32 received at 735169@bugs.debian.org (full text, mbox, reply):

From: Cyril Brulebois <kibi@debian.org>

To: Andreas Henriksson <andreas@fatal.se>, 735169@bugs.debian.org

Cc: kcao <cc.5nth@gmail.com>, Andrii Senkovych <jolly_roger@itblog.org.ua>

Subject: Re: Bug#735169: os-prober cannot work with the latest upstream blkid (util-linux-2.24)

Date: Thu, 8 Oct 2015 02:36:28 +0200

[Message part 1 (text/plain, inline)]

Control: tag -1 pending
Control: severity -1 important

Hi,

Andreas Henriksson <andreas@fatal.se> (2014-07-12):
> Control: tags -1 + patch
> 
> Hello!
> 
> I'm working on an updated util-linux package (v2.24.2) and was pointed
> to this os-prober bug report.
> 
> In my limited testing I haven't been able to reproduce any problems
> here with os-prober and util-linux 2.24.2.
> 
> I can confirm though that the behaviour of running blkid against
> an extended dos partition has changed. In the new version information
> is correctly reported for those instead of just erroring out.
> 
> The old blkid version (2.20.1) returns error code 2 when running
> against an extended dos partition.
> 
> $ sudo blkid -V
> blkid from util-linux 2.24.2  (libblkid 2.24.0, 24-Apr-2014)
> 
> $ sudo blkid -o export /dev/sda2
> DEVNAME=/dev/sda2
> PTUUID=dd6e951b
> PTTYPE=dos
> PARTUUID=000ae69f-02
> 
> $ sudo blkid /dev/sda2
> /dev/sda2: PTUUID="dd6e951b" PTTYPE="dos" PARTUUID="000ae69f-02"
> 
> 
> I'm attaching a patch that should hopefully help. It tries to make the
> executed codepath similar to what it was by adding an extra check on the
> case where value of TYPE is "" and then exits 0 if it looks like an
> extended dos partition.
> 
> Any feedback would be greatly appreciated.

Thanks for the patch, and sorry for the delay. I've pushed it to git
for now (it'll likely be uploaded by Christian when he has time). I'm
adjusting the severity as well as I don't think wishlist is right for
this kind of bug.

A cherry-pick to stable might be a good idea at some point, I suppose
(jessie has 2.25.2).

Mraw,
KiBi.

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 735169-close@bugs.debian.org (full text, mbox, reply):

From: Cyril Brulebois <kibi@debian.org>

To: 735169-close@bugs.debian.org

Subject: Bug#735169: fixed in os-prober 1.68

Date: Thu, 08 Oct 2015 12:36:42 +0000

Source: os-prober
Source-Version: 1.68

We believe that the bug you reported is fixed in the latest version of
os-prober, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 735169@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Cyril Brulebois <kibi@debian.org> (supplier of updated os-prober package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Thu, 08 Oct 2015 14:26:16 +0200
Source: os-prober
Binary: os-prober-udeb os-prober
Architecture: source
Version: 1.68
Distribution: unstable
Urgency: medium
Maintainer: Debian Install System Team <debian-boot@lists.debian.org>
Changed-By: Cyril Brulebois <kibi@debian.org>
Description:
 os-prober  - utility to detect other OSes on a set of drives
 os-prober-udeb - utility to detect other OSes on a set of drives (udeb)
Closes: 735169 801278
Changes:
 os-prober (1.68) unstable; urgency=medium
 .
   * Adjust extended dos partition support for blkid/util-linux 2.24+,
     thanks to Andreas Henriksson (Closes: #735169).
   * Add support for Windows 10 (otherwise reported as Windows Recovery
     Environment). Thanks, Philipp Wolfer! (Closes: #801278).
Checksums-Sha1:
 191b164d90a1e6f2b445862a4705f1a981714f73 1734 os-prober_1.68.dsc
 42b0fbf12647e09296c6771ffb4ff150ec068d5b 25336 os-prober_1.68.tar.xz
Checksums-Sha256:
 31d133792e3e6e7217097ede27d3987bfe8c136b1a5347d29216038c71a002b4 1734 os-prober_1.68.dsc
 ad0b98c3e59239bff94e8f03bb618c21043c5314a7d00726d53f84add8478804 25336 os-prober_1.68.tar.xz
Files:
 403f2732653ccd7b0049b1e2d4f4c80f 1734 debian-installer optional os-prober_1.68.dsc
 11e9794cc4bfd77092c76d9d96a2035b 25336 debian-installer optional os-prober_1.68.tar.xz

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJWFmEyAAoJEP+RSvDCs1UgmIIQAJL4TunQmmXXXFkKoQtBFbvz
cylGLPb0ZJXFgyed4lh6wMPon6aAqZCH3JIITsAFP+CIUaJVmGr5z0SQpo84Mffi
gTSQkkp4+b1X4w3KOM41yyzfmQISBWuJB3Uii5RUpy1VPDj1ZsQcjJompYol0Caz
gV6F21b0a4TgeB4Y3e1aa2PGN0wkWyYuAVx3BAdLl1oNkAJrCcIOQl3EVHKjlJmp
4KLpfPKTL24RO4E6K2EKbXYLlaTg1Ag26qPG75UGI1KjQ0sOmOA49DarvSsMtkqj
9hwGHd1jB8jvYvRn4GaNg8ATMPDs0I+sH+crkGMvvBC5Glm9dwYjNbsr1CZATS+u
MAVIMy733mcMniASfY+x/xuSj2L0hR1CtXW/UkE/ZtLSOTLU2Za0h3EgZFl9NP3f
NvBqak0KDCxIU6ymt2lGJG7NHW92y0ppjBpKS6y2rFOw++29vPxAEEygL8cX4Duc
zgYmt6zeZ3C9qnMRVkbgVwN01nXCdwknWxs1Ia4qLUfghQzy4wGo5GC59KoXFF24
ADokFrHxpRZPLc4Tfy6/7E4C7pL0mIDUKLAjh27XvllwtytbB9FZF5KKDhkCtC1r
nYvJShQAt3kqTOM4GmzXmPBT3wQBkdtyBQzrgJIcAlp48HEpf7VcuhTXw1orpV5Z
y9keXE7cxev+3c0vF32X
=l1jP
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.