Debian Bug report logs - #785512
regression: cannot find iso image on usb stick any more

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Norbert Preining <preining@logic.at>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: regression: cannot find iso image on usb stick any more

Date: Sun, 17 May 2015 19:34:32 +0900

Package: installation-reports
Severity: important

Hi,

I am always carrying around a bootable USB stick with various rescue discs
and an installation image of Debian.

Till about 1 year (6 months?) ago it was possible to boot via
grub/iso the image. The grub entry was
        set isofile="/firmware-testing-amd64-netinst.iso"
        menuentry '64 bit Install' {
          loopback loop (hd0,1)$isofile
          linux    (loop)/install.amd/vmlinuz vga=788 -- quiet
          initrd   (loop)/install.amd/initrd.gz
        }
That worked without a hinch for long time.

The other day I thought about updating this to the current jessie install
image
        set isofile="/firmware-8.0.0-amd64-i386-netinst.iso"
(with both the dual-arch iso as well as the amd64 only iso!)
but the installation process hangs at detecting the CDROM (not detected).

In the syslog I see that it is mounting sdb1 (the usb stick), but
then gives up:
May 16 12:43:54 cdrom-detect: Searching for Debian installation media...
May 16 12:43:54 cdrom-detect: Devices: ''
May 16 12:43:55 cdrom-detect: Devices: ''
May 16 12:43:56 cdrom-detect: Devices: ''
May 16 12:43:57 cdrom-detect: Devices: ''
May 16 12:43:58 cdrom-detect: Devices: ''
May 16 12:43:59 cdrom-detect: Devices: ''
May 16 12:44:00 cdrom-detect: Devices: ''
May 16 12:44:01 cdrom-detect: Devices: ''
May 16 12:44:02 cdrom-detect: Devices: ''
May 16 12:44:03 kernel: [   15.511993] random: nonblocking pool is initialized
May 16 12:44:03 cdrom-detect: CD-ROM mount failed: device=/dev/sdb1
fstype=iso9660
May 16 12:44:03 kernel: [   15.523182] FAT-fs (sdb1): utf8 is not a recommended
IO charset for FAT filesystems, filesystem will be case sensitive!
May 16 12:44:03 cdrom-detect: CD-ROM mount succeeded: device=/dev/sdb1
fstype=vfat
May 16 12:44:03 cdrom-detect: The CD in /dev/sdb1 is not a Debian CD!
May 16 12:44:19 init: starting pid 186, tty '/dev/tty2': '-/bin/sh'
May 16 12:44:31 kernel: [   43.143830] FAT-fs (sdb1): utf8 is not a recommended
IO charset for FAT filesystems, filesystem will be case sensitive!


Are there any hidden things I have to pass to the kernel boot cmd line
to get this working as before?

It is especially disappointing that this is a regression, as I used the
images quite a long time.

Thanks

Norbert

-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (200, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.1.0-rc3+ (SMP w/4 CPU cores; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Message #10 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Philip Hands <phil@hands.com>

To: Norbert Preining <preining@logic.at>, 785512@bugs.debian.org

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Thu, 28 May 2015 15:26:05 +0100

[Message part 1 (text/plain, inline)]

Hi Norbert,

I can confirm that this no longer works (as described).

It looks to me as though we're missing iso-scan and load-iso from the
cdrom target, and that as a result we neither include the loop module,
nor do we attempt to mount the usb stick, nor look for the ISO therein.

One can just use the hd-media kernel and initrd (actually I think one
could get away with just the initrd, and the kernel from the ISO, but
that's probably asking for trouble).

Having just tried that with the vmlinuz and initrd.gz found here:

  http://ftp.uk.debian.org/debian/dists/stable/main/installer-i386/current/images/hd-media/

it gets past the failure point, and starts loading components from the
netinst, so this does work.

It strikes me as a bit of a shame that one cannot just take advantage of
grub's loopback feature to use the files found within the iso image.

Adding the missing components would allow this, so I think that's the
right thing to do unless there's some space constraint that this would
break.

BTW the test I just did ended up doing a "Full Disk Search" before it
found the ISO, so we'd also want to make sure that the ISO is somewhere
that we're hunting for it, or perhaps just specify its location on the
command line (I've not looked into the options to make that work yet).

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/    http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,    GERMANY

[signature.asc (application/pgp-signature, inline)]

Message #15 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Norbert Preining <preining@logic.at>

To: Philip Hands <phil@hands.com>

Cc: 785512@bugs.debian.org

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Fri, 29 May 2015 12:32:13 +0900

Hi Philip,

> It looks to me as though we're missing iso-scan and load-iso from the
> cdrom target, and that as a result we neither include the loop module,
> nor do we attempt to mount the usb stick, nor look for the ISO therein.

Indeed, checking the initrd you mentioned I see the iso-scan and load-iso
packages available, while in the initrd from current image
it is not.

> It strikes me as a bit of a shame that one cannot just take advantage of
> grub's loopback feature to use the files found within the iso image.

Indeed.

> Adding the missing components would allow this, so I think that's the
> right thing to do unless there's some space constraint that this would
> break.

It is strange that the changelog of debian-installer in *not*one*word*
mentiones that iso-scan got lost ... maybe I am hallucinating and
didn't install my laptop in the way I thought I did ;-)

> BTW the test I just did ended up doing a "Full Disk Search" before it
> found the ISO, so we'd also want to make sure that the ISO is somewhere

Not necessary, a simple
	iso-scan/filename=$isofile iso-scan/ask_second_pass=true
added to the cmd line works.

That means, one *only* needs to replace the initrd. My grub.cfg
entries now look like:

    set isofile="/boot/iso/firmware-8.0.0-amd64-i386-netinst.iso"
    menuentry '64 bit Install' {
        set background_color=black
        loopback loop (hd0,1)$isofile
        linux    (loop)/install.amd/vmlinuz iso-scan/ask_second_pass=true iso-scan/filename=$isofile vga=788 -- quiet 
        initrd   /boot/debian/install.amd/initrd.gz
    }

So the kernel is taken from the iso image, only the initrd is replaced.

Probably the safest option is to use *NOT* the initrd from the daily
builds, but from the release.


I will update the blog later on, now I have a meeting.

Big thanks for the information!!!


Norbert

------------------------------------------------------------------------
PREINING, Norbert                               http://www.preining.info
JAIST, Japan                                 TeX Live & Debian Developer
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
------------------------------------------------------------------------

Message #20 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Christian Kastner <debian@kvr.at>

To: Norbert Preining <preining@logic.at>

Cc: 785512@bugs.debian.org, Philip Hands <phil@hands.com>

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Sat, 30 May 2015 11:19:17 +0200

Hi,

On Fri, 29 May 2015 12:32:13 +0900 Norbert Preining <preining@logic.at>
wrote:
> > It looks to me as though we're missing iso-scan and load-iso from the
> > cdrom target, and that as a result we neither include the loop module,
> > nor do we attempt to mount the usb stick, nor look for the ISO therein.
> 
> Indeed, checking the initrd you mentioned I see the iso-scan and load-iso
> packages available, while in the initrd from current image
> it is not.

I believe this issue (or a variant of it) was already reported as #618000.

A fix in the ISO would be great. I used as similar workaround as the one
you mentioned in your blog post, but that quickly became tedious
(assuming one updates the stick at every point release, too).

Regards,
Christian

Message #25 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Brian Potkin <claremont102@gmail.com>

To: Christian Kastner <debian@kvr.at>, 785512@bugs.debian.org

Cc: Norbert Preining <preining@logic.at>, Philip Hands <phil@hands.com>

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Sat, 30 May 2015 20:53:37 +0100

On Sat 30 May 2015 at 11:19:17 +0200, Christian Kastner wrote:

> On Fri, 29 May 2015 12:32:13 +0900 Norbert Preining <preining@logic.at>
> wrote:
> > > It looks to me as though we're missing iso-scan and load-iso from the
> > > cdrom target, and that as a result we neither include the loop module,
> > > nor do we attempt to mount the usb stick, nor look for the ISO therein.
> > 
> > Indeed, checking the initrd you mentioned I see the iso-scan and load-iso
> > packages available, while in the initrd from current image
> > it is not.
> 
> I believe this issue (or a variant of it) was already reported as #618000.

#618000 and this bug are related (both involve the kernel module loop.ko
being in the initrd) but they are not the same issue. #618000 is an
enhancement request for D-I. #724931 could be viewed as an attempt to
move the issue to a conclusion.

On the other hand, #785512 is reporting a "regression" between the
Jessie installer and some random, unknown, testing version for which
documentation of any intentional change is lacking. Note that it is not
a "regression" compared with the Wheezy installer, which is also unable
to take advantage of GRUB's loopback facility.

> A fix in the ISO would be great. I used as similar workaround as the one
> you mentioned in your blog post, but that quickly became tedious
> (assuming one updates the stick at every point release, too).

There are at least three ways to boot a netinst ISO from a USB stick. An
additional one based on GRUB's loopback would be nice. All have their
advantages and disadvantages but installing Debian successfully without
the tedium and in a controlled manner is still possible without it.

Regards,

Brian.

Message #30 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Norbert Preining <preining@logic.at>

To: Brian Potkin <claremont102@gmail.com>

Cc: Christian Kastner <debian@kvr.at>, 785512@bugs.debian.org, Philip Hands <phil@hands.com>

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Mon, 1 Jun 2015 00:51:58 +0900

Hi Brian,

On Sat, 30 May 2015, Brian Potkin wrote:
> On the other hand, #785512 is reporting a "regression" between the
> Jessie installer and some random, unknown, testing version for which
> documentation of any intentional change is lacking. Note that it is not

Well, it is not *completely* unknown, it was this file:
http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/daily-builds/sid_d-i/current/amd64/iso-cd/firmware-testing-amd64-netinst.iso
on/around 2014-05.

So I wouldn't say this is "random", "unknown" etc ...

> a "regression" compared with the Wheezy installer, which is also unable
> to take advantage of GRUB's loopback facility.

It is not about GRUB's loopback facility, but the missing
	iso-scan
package in current installers. It don't know whether this packages
was loaded in wheezy or not, but it was loaded in the above one.

> There are at least three ways to boot a netinst ISO from a USB stick. An

Hmm, I didn't find *one* that leaves the sstick useable for other things,
and allows to boot into a variety of other rescue iso images etc
at the same time. Please let me know which one?

(Fat16 - was that a joke on the current jessie installer images about
needing fat16 on the usb stick? The options explained on
https://www.debian.org/releases/stable/i386/ch04s03.html.en
are all either destructive of previous data, or don't play well as
carry on sticks for the emergency case, nor nor nor. ....)

> additional one based on GRUB's loopback would be nice. All have their
> advantages and disadvantages but installing Debian successfully without
> the tedium and in a controlled manner is still possible without it.

Is there a reason why iso-scan is *not* loaded?

Thanks

Norbert

------------------------------------------------------------------------
PREINING, Norbert                               http://www.preining.info
JAIST, Japan                                 TeX Live & Debian Developer
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
------------------------------------------------------------------------

Message #35 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Brian Potkin <claremont102@gmail.com>

To: Norbert Preining <preining@logic.at>, 785512@bugs.debian.org

Cc: Christian Kastner <debian@kvr.at>, Philip Hands <phil@hands.com>

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Mon, 1 Jun 2015 13:52:09 +0100

On Mon 01 Jun 2015 at 00:51:58 +0900, Norbert Preining wrote:

> Hi Brian,

Hello Norbert,

> On Sat, 30 May 2015, Brian Potkin wrote:
> > On the other hand, #785512 is reporting a "regression" between the
> > Jessie installer and some random, unknown, testing version for which
> > documentation of any intentional change is lacking. Note that it is not
> 
> Well, it is not *completely* unknown, it was this file:
> http://cdimage.debian.org/cdimage/unofficial/non-free/cd-including-firmware/daily-builds/sid_d-i/current/amd64/iso-cd/firmware-testing-amd64-netinst.iso
> on/around 2014-05.
> 
> So I wouldn't say this is "random", "unknown" etc ...
> 
> > a "regression" compared with the Wheezy installer, which is also unable
> > to take advantage of GRUB's loopback facility.
> 
> It is not about GRUB's loopback facility, but the missing
> 	iso-scan
> package in current installers. It don't know whether this packages
> was loaded in wheezy or not, but it was loaded in the above one.

I do not know why a testing image should behave differently from the
realeased Wheezy and Jessie images. I have a recollection I experienced
something similar with a Wheezy testing image but put it down to things
coming and going during the testing period.

> > There are at least three ways to boot a netinst ISO from a USB stick. An
> 
> Hmm, I didn't find *one* that leaves the sstick useable for other things,
> and allows to boot into a variety of other rescue iso images etc
> at the same time. Please let me know which one?

Surely the method you describe on your blog and the use of the hd-media
method allows this? Another method is to partition the stick. The grub
directory and the ISO images to use with loopback can be on the larger
partition. Then dd the firmware ISO to the second partition and boot it
with a stanza like

  menuentry "Debian 8.0.0 i386 netinst with firmware" {
  search --label --set=root "Debian 8.0.0 i386 1"
  linux /install.386/vmlinuz
  initrd /install.386/initrd.gz
  }
 
> (Fat16 - was that a joke on the current jessie installer images about
> needing fat16 on the usb stick? The options explained on
> https://www.debian.org/releases/stable/i386/ch04s03.html.en
> are all either destructive of previous data, or don't play well as
> carry on sticks for the emergency case, nor nor nor. ....)

Unlike ext* modules the fat and vfat modules are available from the very
start of the install.

> > additional one based on GRUB's loopback would be nice. All have their
> > advantages and disadvantages but installing Debian successfully without
> > the tedium and in a controlled manner is still possible without it.
> 
> Is there a reason why iso-scan is *not* loaded?

The author of iso-scan has this to say:

  iso-scan is part of the Debian installer[1].
 
  However, it is only included in the hd-media initrd. There is no reason
  to include it on the regular CD initrd, because isohybrid allows
  mounting the USB stick directly. (Not a loop-mount of an iso file
  included in some disk, which the hd-media initrd handles.)

   https://lists.debian.org/debian-boot/2013/09/msg00097.html

Regards,

Brian.

Message #40 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Christian Kastner <debian@kvr.at>

To: Brian Potkin <claremont102@gmail.com>, Norbert Preining <preining@logic.at>, 785512@bugs.debian.org

Cc: Philip Hands <phil@hands.com>

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Mon, 01 Jun 2015 16:57:23 +0200

On 2015-06-01 14:52, Brian Potkin wrote:
> On Mon 01 Jun 2015 at 00:51:58 +0900, Norbert Preining wrote:
>>> There are at least three ways to boot a netinst ISO from a USB stick. An
>>
>> Hmm, I didn't find *one* that leaves the sstick useable for other things,
>> and allows to boot into a variety of other rescue iso images etc
>> at the same time. Please let me know which one?
> 
> Surely the method you describe on your blog and the use of the hd-media
> method allows this? Another method is to partition the stick. The grub
> directory and the ISO images to use with loopback can be on the larger
> partition. Then dd the firmware ISO to the second partition and boot it
> with a stanza like
> 
>   menuentry "Debian 8.0.0 i386 netinst with firmware" {
>   search --label --set=root "Debian 8.0.0 i386 1"
>   linux /install.386/vmlinuz
>   initrd /install.386/initrd.gz
>   }

Am I wrong in assuming that most of these steps could be avoided if
iso-scan were included in the CD initrd?

> The author of iso-scan has this to say:
> 
>   iso-scan is part of the Debian installer[1].
>  
>   However, it is only included in the hd-media initrd. There is no reason
>   to include it on the regular CD initrd, because isohybrid allows
>   mounting the USB stick directly. (Not a loop-mount of an iso file
>   included in some disk, which the hd-media initrd handles.)
> 
>    https://lists.debian.org/debian-boot/2013/09/msg00097.html

In an age where the cheapest USB drive (at 5€) will get you 16GB, and
10€ will get you a 32GB, with sufficient space for 3 resp. 7 full DVD
images, perhaps this statement should be revisited.

Having a single USB drive with eg multiple Jessie images, say for
various popular architectures, and without having to resort to
hd-media[1], would be nice.

Regards,
Christian

[1] I know this is lazy, but locating and downloading all the relevant
media, and keeping them up-to-date really is tedious to me. Compare this
to GRML or Ubuntu images, for example, where the update process is
basically just copying the ISO to the drive, and maybe perhaps updating
the pathname (of an image) in grub.conf.

Message #45 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Norbert Preining <preining@logic.at>

To: Brian Potkin <claremont102@gmail.com>, Christian Kastner <debian@kvr.at>

Cc: 785512@bugs.debian.org, Philip Hands <phil@hands.com>

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Tue, 2 Jun 2015 08:25:53 +0900

Hi Brian,

On Mon, 01 Jun 2015, Brian Potkin wrote:
> [...] but put it down to things coming and going during the testing period.

Ok, duly noted. (Un)happy coincidence that I used just one of those.

> Surely the method you describe on your blog and the use of the hd-media
> method allows this?

Yes, but is tedious to update/

> Another method is to partition the stick. [...]

Of course, this is an option, but doesn't allow you to have
multiple iso images to be booted (or, ok, unless you make many
partitions).

I agree, there are many methods, but it is about convenience.

> The author of iso-scan has this to say:
> 
>   iso-scan is part of the Debian installer[1].
>  
>   However, it is only included in the hd-media initrd. There is no reason
>   to include it on the regular CD initrd, because isohybrid allows
>   mounting the USB stick directly. (Not a loop-mount of an iso file
>   included in some disk, which the hd-media initrd handles.)
> 
>    https://lists.debian.org/debian-boot/2013/09/msg00097.html

Thanks, I will ask Joey whether I don't want to reconsider this
opinion.

Nowadays laptops practically never come with CD drive, and in many
cases you don't want to go out and buy a new USB stick just for
installation,and you don't want to rescue all your data from the
usb stick before installing Debian.

On Mon, 01 Jun 2015, Christian Kastner wrote:
> In an age where the cheapest USB drive (at 5€) will get you 16GB, and
> 10€ will get you a 32GB, with sufficient space for 3 resp. 7 full DVD
> images, perhaps this statement should be revisited.

Here I agree.

> Having a single USB drive with eg multiple Jessie images, say for
> various popular architectures, and without having to resort to
> hd-media[1], would be nice.

As well as here.

All the best

Norbert

------------------------------------------------------------------------
PREINING, Norbert                               http://www.preining.info
JAIST, Japan                                 TeX Live & Debian Developer
GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
------------------------------------------------------------------------

Message #50 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Philip Hands <phil@hands.com>

To: Brian Potkin <claremont102@gmail.com>, 785512@bugs.debian.org

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Tue, 02 Jun 2015 13:21:40 +0100

[Message part 1 (text/plain, inline)]

Brian Potkin <claremont102@gmail.com> writes:
...
>> Is there a reason why iso-scan is *not* loaded?
>
> The author of iso-scan has this to say:
>
>   iso-scan is part of the Debian installer[1].
>  
>   However, it is only included in the hd-media initrd. There is no reason
>   to include it on the regular CD initrd, because isohybrid allows
>   mounting the USB stick directly. (Not a loop-mount of an iso file
>   included in some disk, which the hd-media initrd handles.)
>
>    https://lists.debian.org/debian-boot/2013/09/msg00097.html

As Joey says, one doesn't need it if one is doing exactly what was
envisioned with the images (i.e. dd-ing straight to the device) but
it is very clear by now that other valid use cases exist.

It is of course possible to work round this lack with the hd-media
files, but there's definitely an advantage to knowing that one is using
the right kernel/initrd, as one does if using the files from within the
image itself.

For that to work those files must include iso-scan.

Cheers, Phil.
-- 
|)|  Philip Hands  [+44 (0)20 8530 9560]  HANDS.COM Ltd.
|-|  http://www.hands.com/    http://ftp.uk.debian.org/
|(|  Hugo-Klemm-Strasse 34,   21075 Hamburg,    GERMANY

[signature.asc (application/pgp-signature, inline)]

Message #55 received at 785512@bugs.debian.org (full text, mbox, reply):

From: "Lennart Sorensen" <lsorense@csclub.uwaterloo.ca>

To: Philip Hands <phil@hands.com>, 785512@bugs.debian.org

Cc: Brian Potkin <claremont102@gmail.com>

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Tue, 2 Jun 2015 12:27:31 -0400

On Tue, Jun 02, 2015 at 01:21:40PM +0100, Philip Hands wrote:
> As Joey says, one doesn't need it if one is doing exactly what was
> envisioned with the images (i.e. dd-ing straight to the device) but
> it is very clear by now that other valid use cases exist.
> 
> It is of course possible to work round this lack with the hd-media
> files, but there's definitely an advantage to knowing that one is using
> the right kernel/initrd, as one does if using the files from within the
> image itself.
> 
> For that to work those files must include iso-scan.

I guess if you have a large USB key, you could dd the first bluray image
to it (assuming it is a 32GB stick).

That should cover most people's package needs.

-- 
Len Sorensen

Message #60 received at 785512@bugs.debian.org (full text, mbox, reply):

From: Christian Kastner <ckk@debian.org>

To: 785512@bugs.debian.org

Subject: Re: Bug#785512: regression: cannot find iso image on usb stick any more

Date: Wed, 13 Mar 2019 19:54:31 +0100

On Thu, 28 May 2015 15:26:05 +0100 Philip Hands <phil@hands.com> wrote:>
It looks to me as though we're missing iso-scan and load-iso from the
> cdrom target, and that as a result we neither include the loop module,
> nor do we attempt to mount the usb stick, nor look for the ISO therein.

This is an ancient request, but as I found myself today again dd'ing
numerous USB sticks, so I thought I'd poke it again:

Would it be possible to get iso-scan (or whatever is needed to get the
installer to find ISO files on a drive) in the CD initrd?

32GB USB sticks go for €4.50 here, it would be great if the space could
be used for multiple ISOs, instead of having to purchase one stick each
per intended ISO.

Regards,
Christian

Send a report that this bug log contains spam.