Debian Bug report logs - #502446
debian-installer: grub-installer is not updated for grub-pc introduction

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Thu, 16 Oct 2008 23:41:22 +0900

Package: debian-installer
Version: rc2
Severity: important

rc2 comes with grub-pc in its initramfs /pool directory.  So d-i team
decided to offer not only grub (grub legacy) but also grub-pc (grub 2).
Good for you.

  grub-installer_1.32_i386.udeb
  grub-pc_1.96+20080512-1_i386.deb
  grub_0.97-39_i386.deb 

But if you do, you must take care critical diffrences of these 2
programs.

For /dev/hda3, grub legacy used (hd0,2) while grub 2 use (hd0,3).

The template file in grub-installer needed to be updated not to mislead
users.  Template: grub-installer/bootdev needs to be updated 
 "(hd0,1)" --> "(hd0,2)"
 "(hd2,4)" --> "(hd2,5)"
as:

Template: grub-installer/bootdev
Type: string
Description: Device for boot loader installation:
 You need to make the newly installed system bootable, by installing
 the GRUB boot loader on a bootable device. The usual way to do this is to
 install GRUB on the master boot record of your first hard drive. If you
 prefer, you can install GRUB elsewhere on the drive, or to another drive,
 or even to a floppy.
 .
 The device can be specified using GRUB's "(hdn,m)" notation, or as a device
 in /dev. Below are some examples:
  - "(hd0)" or "/dev/hda" will install GRUB to the master boot record
    of your first hard drive (IDE);
  - "(hd0,2)" or "/dev/hda2" will use the second partition of your
    first IDE drive;
  - "(hd2,5)" or "/dev/sdc5" will use the first extended partition of
    your third drive (SCSI here);
  - "(fd0)" or "/dev/fd0" will install GRUB to a floppy.

This should be easy to do for all languages and reduce confusion.

Question is how to impliment it without breaking grub legacy.  Maybe you
create "Template: grub2-installer/bootdev" and adjust script based on
boot loader used.

I think this is one of the root cause of problem people are reporting
such as: 501942.

The other issue may be around 30_os-prober script in grub-pc package.  I
need to check more but this is less problematic since at least main OS
will boot from right boot sector.  

(If I am pedantic, I can say that "/dev/sdc5" may not be the first
extended partition but simply 5th partition in GPT.  But this issue can
be ignored since it is unlikely to cause any major cofusion.)

-- System Information:
Debian Release: lenny/sid
  APT prefers unstable
  APT policy: (800, 'unstable'), (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.26-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Message #12 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: 502446@bugs.debian.org

Subject: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Fri, 17 Oct 2008 22:35:48 +0900

Hi,

I guess it is now reassigned grub-installer.

Not just my emperical experience, here is rational for this bug report.

  http://grub.enbug.org/grub.cfg
---
NOTE: you can use the update-grub utility to generate a grub.cfg file
for you

GRUB 2 uses new bashish scripting language that is used to build menus.
At startup file grub.cfg will be read and executed.

Please note that partition numbering has been changed to start from 1
(in grub legacy it started from 0). 
---

As I looked around, here is other issues wth grub-pc install in rc2.

Issuing grub-install /dev/sd? with right install device did not fix
my situation completely.

Editting /boot/grub/grub.cfg with right root device for booting fixed the
situation.

Since this is generated file, root cause is its generation.  Generation script
is update-grub.  Its /etc/grub.d/30_os-prober is the source of the problem.  It
calls commands packaged in os-prober.  Most related parts of os-prober is its
backend script:

/usr/lib/linux-boot-probes/mounted/40grub
/usr/lib/linux-boot-probes/mounted/40grub2

Since all these scripts seems to assume that grub legacy and grub 2 to use the
same device name,  they are in trouble and needs to be fixed in sync.

I also do not know how we should deal with this at this late stage of release. 

I hope to get some response from maintainers involved.  (os-prober is also d-i
team pakage)

If time is too late, we should document it in release note at least.

To wrap up:

Overwriting unrelated partition's boot sector qualifies as "critical" bug since
this makes unrelated software (i.e. other OS's boot sector) on the system break
where you install the package.

Other issue like having broken multiboot is just annoiance.

For now, I keep this as "important" bug.

Osamu

Message #17 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: 502446@bugs.debian.org

Cc: control@bugs.debian.org

Subject: Patch for Bug#502446: grub-installer is not updated for grub-pc introduction

Date: Sun, 19 Oct 2008 12:15:27 +0900

[Message part 1 (text/plain, inline)]

tags 502446 patch
thanks

Just get my point clear, I am submitting untested patch as reference.

Once tested to be good, it is technically trivial (but time consuming)
to add po files for most European languages and CJK as long as po files
are in UTF-8.  So this will not make much regression.

My work on rescue scripts are pure guessing work (but simple one.)

Osamu

[grub-installer-1.35-fix.diff (text/x-diff, attachment)]

Message #24 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: 502446@bugs.debian.org

Subject: patch second try... Bug#502446: grub-installer

Date: Tue, 21 Oct 2008 23:29:00 +0900

[Message part 1 (text/plain, inline)]

Hi,

For Bug #502446: http://bugs.debian.org/502446

Here is updated patch which takes care odd partition offset of grub
legacy.  Around frdev, I may have been too aggressive....

I will test this later ...

Osamu

[grub-installer-1.35.2ndtry.diff (text/x-diff, attachment)]

Message #29 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: 502446@bugs.debian.org

Subject: Bug#502446: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Sat, 1 Nov 2008 00:59:48 +0900

Hi,

As I looked more in detail, debian-installer works fine (does not over
write unrelated data) and usable if MBR is used as BIOS emultion mode
boot loader. (I.e., installed to /dev/sda (hd0)

So problem is when installing to /dev/sda3 or in other word to (hd0,3).

Since this causes data loss, this could be considered grave.  But
changing d-i this late may not be good idea.  If so reminder not to
install to places such as /dev/sda3.

Anyway, installation needs to do few manual tricks including gptsync.

  http://wiki.debian.org/MacBook/DebianInstallTutorial

Documenting this as short comments in release note is good idea.

Osamu

Message #34 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: 502446@bugs.debian.org

Subject: Bug#502446: grub-installer patch with complete po file updates :-)

Date: Tue, 4 Nov 2008 01:53:59 +0900

[Message part 1 (text/plain, inline)]

Hi,

Current grub-installer script will overwrite boot sector of wrong
partition for Intel based Mac users.  This is not so nice thing to have
for release.

I now have tested and updated patch for this bug and test d-i image at:
 http://people.debian.org/~osamu/d-i-osamu-grub2.iso
 http://people.debian.org/~osamu/grub-installer_1.35.all-but-po.diff.gz

It works!

I also made full patch including translated text for all languages and
put at: 
  http://people.debian.org/~osamu/grub-installer_1.35.all.diff.gz 
      (No code change but just po and comment out of set -x.)

So this patch will not degrade translation status, and please consider
this fr inclusion.  Patch is based on unstable 1.35 package.

---
Here is details.

Since my bug report and my patch concept, I spend some time to recall
d-i testing tricks.  (I made diunpk and dipk script:
http://wiki.debian.org/DebianInstaller/Modify/CD )

My initial patch had typo but my idea was confirmed right.
 1) GRUB2 device name has no offset and need adjustment as proposed.
 2) rescue mode now offer boot loader installation on GRUB 2 too.
 3) 30_os-prober script of grub-pc is fine but it must be run after
    rebooting system to be more useful. (see below)
 4) gptsync needs to be installed and run from shell console in chroot
    but this can be done by rEFIt chain bootloader side too.  Thus d-i
    is usable on MacTel (Intel Macs).

    I intentionally left not-so-precise statement of 
    "first extended partition" under GPT.  This is because 
    this will create translation headache.

Osamu 

FYI: If os-prober package is initially installed and a post reboot hook
     script is made to run update-grub script, grub.cfg is generated
     correctly without 30_otheros from grub-installer.  This will make
     cleaner installation code.  This should be post-lenny thing.

[grub-installer_1.35.all-but-po.diff.gz (application/octet-stream, attachment)]

Message #39 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: 502446@bugs.debian.org

Subject: update on grub-installer

Date: Wed, 5 Nov 2008 23:37:41 +0900

As I check after good night sleep, I realize my previous patch should
have been not:

+frgrubroot=$(($frbootpart - $grub_part_offset))

but should have been:

+frgrubroot=$(($frgrubroot - $grub_part_offset))

if this should also work for Serial ATA multipath which I do not use.
(RAID should have been OK.)

Other than this, I like Robert Millan's patch and integrated it to mine
which address more issues.

I still think making 30_otheros is ugly hack and if possible should be
replaced with grub's code itself.

After installing os-prober with "apt-install os-prober" , running

$chroot $ROOT update-grub

should create entry but it doesn't as now.  It could likely be
availability of /dev/hd? /dev/sd? etc in chroot.

Osamu

PS: bug in my earlier patch thought was: s/grub_vrsion/grub_version/
+case $grub_vrsion in
+       grub)   frgrubroot=$(($frbootpart - 1)) ;;
+       grub2)  frgrubroot=$frbootpart ;;
+       *)      exit 1 ;;
+esac

This part again hit me when I changed style.

Message #42 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Robert Millan <rmh@aybabtu.com>

To: Osamu Aoki <osamu@debian.org>, 502446@bugs.debian.org

Subject: Re: Bug#502446: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Sat, 15 Nov 2008 12:26:43 +0100

Hi,

Otavio asked me to comment on this.

Notice that I maintain grub/grub2, not grub-installer. In fact, my
recommendations as grub maintainer are often dismissed here, so take
this with a grain of salt.

On Thu, Oct 16, 2008 at 11:41:22PM +0900, Osamu Aoki wrote:
> 
> But if you do, you must take care critical diffrences of these 2
> programs.
> 
> For /dev/hda3, grub legacy used (hd0,2) while grub 2 use (hd0,3).
> 
> The template file in grub-installer needed to be updated not to mislead
> users.  Template: grub-installer/bootdev needs to be updated 
>  "(hd0,1)" --> "(hd0,2)"
>  "(hd2,4)" --> "(hd2,5)"
> as:

It's not critical, and I don't think it's worth it.  It just affects a template
description, and this particular problem is hopefully going away post-Lenny as
soon as GRUB Legacy is completely deprecated.

#473401 OTOH has a real effect in end usability, and should be taken into
consideration for Lenny.

>  The device can be specified using GRUB's "(hdn,m)" notation, or as a device
>  in /dev. Below are some examples:
>   - "(hd0)" or "/dev/hda" will install GRUB to the master boot record
>     of your first hard drive (IDE);

This paragraph leads user to think there's a 1:1 correspondence between GRUB
drives (which are enumerated by BIOS) and Linux devices (which are enumerated
by Linux itself).  However, it's really a game of gambling.  E.g. "/dev/hda"
is "(hd0)" _most of the time_, but not always.

This is something we already solved in GRUB 2 (by using UUIDs).

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Message #47 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: Robert Millan <rmh@aybabtu.com>, 502446@bugs.debian.org

Subject: Re: Bug#502446: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Sun, 16 Nov 2008 19:36:52 +0900

Hi,

On Sat, Nov 15, 2008 at 12:26:43PM +0100, Robert Millan wrote:
> 
> Hi,
> 
> Otavio asked me to comment on this.
> 
> Notice that I maintain grub/grub2, not grub-installer. In fact, my
> recommendations as grub maintainer are often dismissed here, so take
> this with a grain of salt.

Thanks.
 
> On Thu, Oct 16, 2008 at 11:41:22PM +0900, Osamu Aoki wrote:
> > 
> > But if you do, you must take care critical diffrences of these 2
> > programs.
> > 
> > For /dev/hda3, grub legacy used (hd0,2) while grub 2 use (hd0,3).
> > 
> > The template file in grub-installer needed to be updated not to mislead
> > users.  Template: grub-installer/bootdev needs to be updated 
> >  "(hd0,1)" --> "(hd0,2)"
> >  "(hd2,4)" --> "(hd2,5)"
> > as:
> 
> It's not critical, and I don't think it's worth it.  It just affects a template
> description, and this particular problem is hopefully going away post-Lenny as
> soon as GRUB Legacy is completely deprecated.

I agree it is not a critical bug since if grub-pc is installed to MBR
per default, this problem be will not be seen.  

Problem is not just description inconsistency.  If you chose to install
to (hd0,2) or /dev/hda3, grub-pc is currently installed to /dev/hda2
overwriting whatever is there.  This was not something I appreciated.
It may kill some existig boot data.  That is not so nice thing to do.
That is why I was concerned.

I wonder why we offer to install grub-pc to non-MBR locations in lenny.
For me, disabling that is an option. But fixing it is even nicer. 

My patch is updated and factorized:  http://people.debian.org/~osamu/

I now use Robert's fix as a part since I think that is better fix for
the problem.

My string updates covers all language.  As far as lenny is concerned, we
are supporting GRUB Legacy and 2 for different case.  Neglecting this
fact is not good for lenny.  But I am not pushing this to delay release
nor pushing this for squeeze.

> #473401 OTOH has a real effect in end usability, and should be taken into
> consideration for Lenny.

I agree it is quite annoying.  This has to be fixed more.  I do not expect
users has to use interactive mode to edit grub menu. His patch makes
sense too.
 
> >  The device can be specified using GRUB's "(hdn,m)" notation, or as a device
> >  in /dev. Below are some examples:
> >   - "(hd0)" or "/dev/hda" will install GRUB to the master boot record
> >     of your first hard drive (IDE);
> 
> This paragraph leads user to think there's a 1:1 correspondence between GRUB
> drives (which are enumerated by BIOS) and Linux devices (which are enumerated
> by Linux itself).  However, it's really a game of gambling.  E.g. "/dev/hda"
> is "(hd0)" _most of the time_, but not always.

I agree.

> This is something we already solved in GRUB 2 (by using UUIDs).

I wonder if GRUB2's /etc/grub.d/30_os-prober can be used from
grub-installer in squeeze.  I guess chroot does not have apropriate
devices to get this script to produce right result.

Osamu

Message #50 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Robert Millan <rmh@aybabtu.com>

To: Osamu Aoki <osamu@debian.org>

Cc: 502446@bugs.debian.org

Subject: Re: Bug#502446: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Sun, 16 Nov 2008 12:40:12 +0100

On Sun, Nov 16, 2008 at 07:36:52PM +0900, Osamu Aoki wrote:
> 
> I wonder why we offer to install grub-pc to non-MBR locations in lenny.
> For me, disabling that is an option. But fixing it is even nicer. 

Erm, we shouldn't be doing this.  When installing in a partition head, you
can't really check if there will be enough space.  It depends on how much is
reserved by your filesystem of choice, which is often not enough anyway.

And even if it is, it's an ugly hack to reuse a partition for two different
things at the same time.

> > This is something we already solved in GRUB 2 (by using UUIDs).
> 
> I wonder if GRUB2's /etc/grub.d/30_os-prober can be used from
> grub-installer in squeeze.

Should be no problem.  It just takes installing os-prober in the target
system before update-grub is run.

> I guess chroot does not have apropriate
> devices to get this script to produce right result.

I don't think so.  If devices were missing, grub-probe / grub-setup wouldn't
work in this environment, and installation wouldn't be possible.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Message #55 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Osamu Aoki <osamu@debian.org>

To: Robert Millan <rmh@aybabtu.com>

Cc: 502446@bugs.debian.org

Subject: Re: Bug#502446: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Mon, 17 Nov 2008 00:09:56 +0900

Hi,

Thanks for interesting comment.

On Sun, Nov 16, 2008 at 12:40:12PM +0100, Robert Millan wrote:
> On Sun, Nov 16, 2008 at 07:36:52PM +0900, Osamu Aoki wrote:
> > 
> > I wonder why we offer to install grub-pc to non-MBR locations in lenny.
> > For me, disabling that is an option. But fixing it is even nicer. 
> 
> Erm, we shouldn't be doing this.  When installing in a partition head, you
> can't really check if there will be enough space.  It depends on how much is
> reserved by your filesystem of choice, which is often not enough anyway.
> 
> And even if it is, it's an ugly hack to reuse a partition for two different
> things at the same time.

Hmmm...  interesting thought.  If this is true, I agree we should drop
asking non MBR location.

But many Linux distros allow us to install boot loader in chain-loading
mode at MBR or head of partition of ext2/3 as I remember.  

I thought for modern HDD with LBA, there is always first track = 63
sectors = 31.5KiB available in MBR and partition head (formatted with
ext2/ext3/vfat/... at least) which grub assumes to use for GRUB Stage
1.5.  (I know there were days when I envyed HDD with 10 MB capacity and
only 17 sector per track.   Then grub should have problem istalling it
to it.)
 
> > > This is something we already solved in GRUB 2 (by using UUIDs).
> > 
> > I wonder if GRUB2's /etc/grub.d/30_os-prober can be used from
> > grub-installer in squeeze.
> 
> Should be no problem.  It just takes installing os-prober in the target
> system before update-grub is run.

I thought so too and tried it.  But as I tried update-grub command in
chroot at /target while having os-prober installed in chroot from
installer, it did not find other partitions.  /chroot/dev directory did
not have /chroot/dev/sda etc.   /dev/sda etc. were there but could not
be seen from chroot.

> > I guess chroot does not have appropriate
> > devices to get this script to produce right result.
> 
> I don't think so.  If devices were missing, grub-probe / grub-setup wouldn't
> work in this environment, and installation wouldn't be possible.

Device was not missing.  /dev/sda exists.  But issuing chroot /target
without appropriate care in advance caused problem.  Thus there is need
to write code :-)  

Osamu

Message #60 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Frans Pop <elendil@planet.nl>

To: Osamu Aoki <osamu@debian.org>, 502446@bugs.debian.org

Subject: Re: Bug#502446: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Sun, 16 Nov 2008 17:09:25 +0100

On Sunday 16 November 2008, Osamu Aoki wrote:
> I thought so too and tried it.  But as I tried update-grub command in
> chroot at /target while having os-prober installed in chroot from
> installer, it did not find other partitions.  /chroot/dev directory did
> not have /chroot/dev/sda etc.   /dev/sda etc. were there but could not
> be seen from chroot.

The installer does, as part of base-installer:
mount /dev /target/dev -o bind

Message #63 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Robert Millan <rmh@aybabtu.com>

To: Osamu Aoki <osamu@debian.org>

Cc: 502446@bugs.debian.org

Subject: Re: Bug#502446: debian-installer: grub-installer is not updated for grub-pc introduction

Date: Mon, 17 Nov 2008 16:54:28 +0100

On Mon, Nov 17, 2008 at 12:09:56AM +0900, Osamu Aoki wrote:
> On Sun, Nov 16, 2008 at 12:40:12PM +0100, Robert Millan wrote:
> > On Sun, Nov 16, 2008 at 07:36:52PM +0900, Osamu Aoki wrote:
> > > 
> > > I wonder why we offer to install grub-pc to non-MBR locations in lenny.
> > > For me, disabling that is an option. But fixing it is even nicer. 
> > 
> > Erm, we shouldn't be doing this.  When installing in a partition head, you
> > can't really check if there will be enough space.  It depends on how much is
> > reserved by your filesystem of choice, which is often not enough anyway.
> > 
> > And even if it is, it's an ugly hack to reuse a partition for two different
> > things at the same time.
> 
> Hmmm...  interesting thought.  If this is true, I agree we should drop
> asking non MBR location.
> 
> But many Linux distros allow us to install boot loader in chain-loading
> mode at MBR or head of partition of ext2/3 as I remember.  
> 
> I thought for modern HDD with LBA, there is always first track = 63
> sectors = 31.5KiB available in MBR and partition head (formatted with
> ext2/ext3/vfat/... at least) which grub assumes to use for GRUB Stage
> 1.5.  (I know there were days when I envyed HDD with 10 MB capacity and
> only 17 sector per track.   Then grub should have problem istalling it
> to it.)

The ~32 kiB after MBR can be assumed to be present, since all but very rare
partitioning software will leave that gap.  This doesn't depend on the
filesystem, as it's a partitioning issue.

What each filesystem reserves in its partition head, depends entirely on the
filesystem implementation.  GRUB doesn't use heuristics to determine if it'll
fit, and it doesn't make any promise about that.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Message #68 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Miguel Figueiredo <elmig@debianpt.org>

To: 502446@bugs.debian.org

Date: Sun, 28 Nov 2010 12:46:28 +0000

Hi all,

since 2 years ago, when this bug had activity, there were many improvements 
regarding grub-pc support in grub-installer.

Can you comment if this bug can be closed or if it's still useful to have it 
open?
 
-- 
Melhores cumprimentos/Best regards,

Miguel Figueiredo
http://www.DebianPT.org

Message #73 received at 502446@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: Miguel Figueiredo <elmig@debianpt.org>

Cc: 502446@bugs.debian.org

Subject: Re: your mail

Date: Sun, 28 Nov 2010 17:44:54 +0000

On Sun, Nov 28, 2010 at 12:46:28PM +0000, Miguel Figueiredo wrote:
> since 2 years ago, when this bug had activity, there were many improvements 
> regarding grub-pc support in grub-installer.
> 
> Can you comment if this bug can be closed or if it's still useful to have it 
> open?

It's easy to look at the current templates file and see that this bug
has not been fixed, and should stay open.

(I keep meaning to fix it but any time I remember about it we're in a
string freeze.  Remind me after the squeeze release, perhaps?)

-- 
Colin Watson                                       [cjwatson@debian.org]

Message #78 received at 502446-done@bugs.debian.org (full text, mbox, reply):

From: Miguel Figueiredo <elmig@debianpt.org>

To: 502446-done@bugs.debian.org

Date: Sat, 01 Jan 2011 11:56:04 +0000

fixed in grub-installer 1.58.
See #581687

-- 
Melhores cumprimentos/Best regards,

Miguel Figueiredo
http://www.DebianPT.org

Send a report that this bug log contains spam.