Debian Bug report logs - #417407
os-prober: protect partitions with "blockdev --setro"

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Peter Nuttall <p.s.nuttall@dur.ac.uk>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: debian-installer: d-i destroyed existing raid device

Date: Mon, 02 Apr 2007 17:33:14 +0100

[Message part 1 (text/plain, inline)]

Package: debian-installer
Severity: critical
Justification: causes serious data loss


Hello,

I upgraded this morning by downloading the business card image and 
booting off it. I had / on an IDE disk, and /home on a raid1 device made 
up of two sata disks. On upgrade, it was my intention to let d-i 
partition the IDE disk, but hang on to /home. It appeared to do this. 
but reboot, I installed mdadm and attempted to assemble and mount 
/dev/md0. mdadm said that sdb was out of sync, and the kernel said that 
the device lacked superblocks. Testing with fsck -b showed that none of 
the backup superblocks was working.

D-I questions: I just pressed enter at the prompt, and set my keyboard 
and location to enGB. On reaching the disk stage, I asked it to set up 
hda as a LVM with / /home /tmp and /var on seperate paritions. On 
checking the result seemed to suggest no changes to /dev/sda or 
/dev/sdb. I have attached syslog and partman from /var/log/installer. 

If you have any questions, or I can help further, please ask. I'm psn on 
oftc. Any suggestions as to what I did wrong or how to recover would be 
really nice.

Thanks

Pete

-- System Information: Debian Release: 4.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.18-4-k7
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)

[partman.gz (application/octet-stream, attachment)]

[syslog.gz (application/octet-stream, attachment)]

Message #8 received at 417407@bugs.debian.org (full text, mbox, reply):

From: martin f krafft <madduck@debian.org>

To: Peter Nuttall <p.s.nuttall@dur.ac.uk>, 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Mon, 2 Apr 2007 18:45:46 +0200

[Message part 1 (text/plain, inline)]

also sprach Peter Nuttall <p.s.nuttall@dur.ac.uk> [2007.04.02.1833 +0200]:
> If you have any questions, or I can help further, please ask. I'm psn on 
> oftc. Any suggestions as to what I did wrong or how to recover would be 
> really nice.

Please also pass the output of

  /usr/share/bug/mdadm/script 3>&1

run as root. Note that RAID superblocks and fsck -b have nothing to
do with each other.

Do you have backups?

Did you do anything else to the partitions?

What does mdadm -E output for /dev/sd[ab]1 ?

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

[signature.asc (application/pgp-signature, inline)]

Message #13 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Peter Nuttall <p.s.nuttall@durham.ac.uk>

To: 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Mon, 2 Apr 2007 18:22:49 +0100

[Message part 1 (text/plain, inline)]

On Mon, Apr 02, 2007 at 06:45:46PM +0200, martin f krafft wrote:
> also sprach Peter Nuttall <p.s.nuttall@dur.ac.uk> [2007.04.02.1833 +0200]:
> > If you have any questions, or I can help further, please ask. I'm psn on 
> > oftc. Any suggestions as to what I did wrong or how to recover would be 
> > really nice.
> 
> Please also pass the output of
> 
>   /usr/share/bug/mdadm/script 3>&1
> 
> run as root. Note that RAID superblocks and fsck -b have nothing to
> do with each other.

Sorry, the superblocks in question are the ext3 superblocks. I have
attached the output.  

> 
> Do you have backups?  
>

Yes, but not of stuff I have been doing for the last week or so. There
are also large files that I don't have backups of, simply because I
don't have the space. So I can cope without recovery, but it would be
very nice to recover.


> Did you do anything else to the partitions?
> 

I don't recall doing anything else. 

> What does mdadm -E output for /dev/sd[ab]1 ?
> 

Attached.

Thanks for getting back to me so fast, 

Pete

> -- 
>  .''`.   martin f. krafft <madduck@debian.org>
> : :'  :  proud Debian developer, author, administrator, and user
> `. `'`   http://people.debian.org/~madduck - http://debiansystem.info
>   `-  Debian - when you have better things to do than fixing systems

[mdadm.txt (text/plain, attachment)]

[sda.txt (text/plain, attachment)]

[sdb.txt (text/plain, attachment)]

Message #18 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Frans Pop <elendil@planet.nl>

To: Peter Nuttall <p.s.nuttall@dur.ac.uk>, 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Mon, 02 Apr 2007 20:07:35 +0200

[Message part 1 (text/plain, inline)]

On Monday 02 April 2007 18:33, Peter Nuttall wrote:
> D-I questions: I just pressed enter at the prompt, and set my keyboard
> and location to enGB. On reaching the disk stage, I asked it to set up
> hda as a LVM with / /home /tmp and /var on seperate paritions. On
> checking the result seemed to suggest no changes to /dev/sda or
> /dev/sdb. I have attached syslog and partman from /var/log/installer.

This seems correct to me.  Guided partitioning on hda should in no way
affect the two SATA disks, and the partman log at least shows they are
unchanged at the end.

At start of partitioning
========================
parted_server: OUT: 1	32256-300066439679	300066407424	primary	ext3	/dev/scsi/host1/bus0/target0/lun0/part1	
parted_server: OUT: 1	32256-300066439679	300066407424	primary	ext3	/dev/scsi/host3/bus0/target0/lun0/part1	

At end of partitioning
========================
parted_server: OUT: 1	32256-300066439679	300066407424	primary	ext3	/dev/scsi/host1/bus0/target0/lun0/part1	
parted_server: OUT: 1	32256-300066439679	300066407424	primary	ext3	/dev/scsi/host3/bus0/target0/lun0/part1	

I do similar partitioning myself sometimes and have never seen changes
to another disk than the one selected.

the next possibility I considered that the installation of the boot loader
could have gone wrong: that grub could have thought that one of the SATA
disks was "the first disk in the system" instead of the IDE disk. However,
the syslog shows that went correctly as well:
Apr  2 11:04:17 grub-installer: info: Installing grub on '(hd0)'
[...]
Apr  2 11:04:25 grub-installer: (hd0)^I/dev/hda
Apr  2 11:04:25 grub-installer: (hd1)^I/dev/sda
Apr  2 11:04:25 grub-installer: (hd2)^I/dev/sdb

(And even if the wrong disk was selected, I don't think it would have
affected the RAID.)

However, the syslog does show some strange activity while the installer is
probing for other operating systems:
Apr  2 10:45:49 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/discs/disc1/part1
Apr  2 10:45:48 kernel: EXT2-fs: sdb1: couldn't mount because of unsupported optional features (4).
Apr  2 10:45:48 kernel: EXT3-fs: INFO: recovery required on readonly filesystem.
Apr  2 10:45:48 kernel: EXT3-fs: write access will be enabled during recovery.
Apr  2 10:45:48 kernel: kjournald starting.  Commit interval 5 seconds
Apr  2 10:45:48 kernel: EXT3-fs: recovery complete.
Apr  2 10:45:48 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Apr  2 10:45:48 50mounted-tests: debug: mounted as ext3 filesystem
[...]
Apr  2 10:45:49 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/discs/disc1/part1
Apr  2 10:45:49 kernel: EXT2-fs: sda1: couldn't mount because of unsupported optional features (4).
Apr  2 10:45:49 kernel: EXT3-fs: INFO: recovery required on readonly filesystem.
Apr  2 10:45:49 kernel: EXT3-fs: write access will be enabled during recovery.
Apr  2 10:45:49 kernel: kjournald starting.  Commit interval 5 seconds
Apr  2 10:45:49 kernel: EXT3-fs: recovery complete.
Apr  2 10:45:49 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Apr  2 10:45:49 50mounted-tests: debug: mounted as ext3 filesystem
[...]
Apr  2 11:04:14 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/discs/disc1/part1
Apr  2 11:04:14 kernel: EXT2-fs warning (device sda1): ext2_fill_super: mounting ext3 filesystem as ext2
Apr  2 11:04:14 50mounted-tests: debug: mounted as ext2 filesystem
[...]
Apr  2 11:04:14 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/discs/disc2/part1
Apr  2 11:04:14 kernel: EXT2-fs warning (device sdb1): ext2_fill_super: mounting ext3 filesystem as ext2
Apr  2 11:04:14 50mounted-tests: debug: mounted as ext2 filesystem

I have no idea what this means exactly. I suggest asking a file system
expert look at this.

The only advise I can give ATM is not to panic and not to take hasty actions.

Cheers,
FJP

[Message part 2 (application/pgp-signature, inline)]

Message #23 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Jim Paris <jim@jtan.com>

To: Frans Pop <elendil@planet.nl>, 417407@bugs.debian.org

Cc: Peter Nuttall <p.s.nuttall@dur.ac.uk>

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Mon, 2 Apr 2007 14:45:42 -0700

Frans Pop wrote:
> However, the syslog does show some strange activity while the installer is
> probing for other operating systems:
[...]
> Apr  2 10:45:48 kernel: EXT2-fs: sdb1: couldn't mount because of unsupported optional features (4).
> Apr  2 10:45:48 kernel: EXT3-fs: INFO: recovery required on readonly filesystem.
> Apr  2 10:45:48 kernel: EXT3-fs: write access will be enabled during recovery.
[...]
> Apr  2 10:45:49 kernel: EXT2-fs: sda1: couldn't mount because of unsupported optional features (4).
> Apr  2 10:45:49 kernel: EXT3-fs: INFO: recovery required on readonly filesystem.
> Apr  2 10:45:49 kernel: EXT3-fs: write access will be enabled during recovery.

If those two disks were part of a RAID mirror and they were written to
individually, they could end up inconsistent, which may have caused
problems with the RAID later?

-jim

Message #26 received at 417407@bugs.debian.org (full text, mbox, reply):

From: martin f krafft <madduck@debian.org>

To: Peter Nuttall <p.s.nuttall@durham.ac.uk>, 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Mon, 2 Apr 2007 23:46:02 +0200

[Message part 1 (text/plain, inline)]

Can you force-start the degraded array, using something like

  mdadm --assemble --auto=yes --force /dev/md0 /dev/sd[ab]1
  mdadm --run /dev/md0

?

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

[signature.asc (application/pgp-signature, inline)]

Message #31 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Peter Nuttall <p.s.nuttall@durham.ac.uk>

To: 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Mon, 2 Apr 2007 23:20:32 +0100

On Mon, Apr 02, 2007 at 11:46:02PM +0200, martin f krafft wrote:
> Can you force-start the degraded array, using something like
> 
>   mdadm --assemble --auto=yes --force /dev/md0 /dev/sd[ab]1
>   mdadm --run /dev/md0
> 
> ?
> 
> -- 
>  .''`.   martin f. krafft <madduck@debian.org>
> : :'  :  proud Debian developer, author, administrator, and user
> `. `'`   http://people.debian.org/~madduck - http://debiansystem.info
>   `-  Debian - when you have better things to do than fixing systems

mdadm --assemble /dev/md0 works, as does your command above. The array
is fine after resyncing, its just its contents that are gone. 

Pete

Message #34 received at 417407@bugs.debian.org (full text, mbox, reply):

From: martin f krafft <madduck@debian.org>

To: Peter Nuttall <p.s.nuttall@durham.ac.uk>, 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 08:53:24 +0200

[Message part 1 (text/plain, inline)]

also sprach Peter Nuttall <p.s.nuttall@durham.ac.uk> [2007.04.03.0020 +0200]:
> mdadm --assemble /dev/md0 works, as does your command above. The
> array is fine after resyncing, its just its contents that are
> gone. 

I honestly have no idea what could have gone wrong. Can you check
whether the array's UUID (mdadm -D /dev/md0) was changed or whether
it's still the same as before the disaster? /etc/mdadm/mdadm.conf
should list it.

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

[signature.asc (application/pgp-signature, inline)]

Message #39 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Samuel Thibault <samuel.thibault@ens-lyon.org>

To: Frans Pop <elendil@planet.nl>

Cc: Peter Nuttall <p.s.nuttall@dur.ac.uk>, 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 10:20:56 +0200

Hi,

Frans Pop, le Mon 02 Apr 2007 20:07:35 +0200, a écrit :
> Apr  2 10:45:49 os-prober: debug: running /usr/lib/os-probes/50mounted-tests on /dev/discs/disc1/part1
> Apr  2 10:45:48 kernel: EXT2-fs: sdb1: couldn't mount because of unsupported optional features (4).
> Apr  2 10:45:48 kernel: EXT3-fs: INFO: recovery required on readonly filesystem.
> Apr  2 10:45:48 kernel: EXT3-fs: write access will be enabled during recovery.
> Apr  2 10:45:48 kernel: kjournald starting.  Commit interval 5 seconds
> Apr  2 10:45:48 kernel: EXT3-fs: recovery complete.
> Apr  2 10:45:48 kernel: EXT3-fs: mounted filesystem with ordered data mode.
> Apr  2 10:45:48 50mounted-tests: debug: mounted as ext3 filesystem

This looks to me like the culprit: for trying to probe other OSes,
read-only mounts are performed. But here it looks like os-prober didn't
know about the RAID array, and tried to mount devices separately, the
result is that the EXT3-fs driver thinks it is a bit broken, and tries
to _fix_ it, i.e. write things on the volume!

Unfortunately, there doesn't seem to be any option for disabling that:
ext3 disables it only if the device itself is read-only...

Samuel

Message #42 received at 417407@bugs.debian.org (full text, mbox, reply):

From: martin f krafft <madduck@debian.org>

To: Jim Paris <jim@jtan.com>

Cc: Frans Pop <elendil@planet.nl>, 417407@bugs.debian.org, Peter Nuttall <p.s.nuttall@dur.ac.uk>

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 10:28:24 +0200

[Message part 1 (text/plain, inline)]

also sprach Jim Paris <jim@jtan.com> [2007.04.02.2345 +0200]:
> If those two disks were part of a RAID mirror and they were
> written to individually, they could end up inconsistent, which may
> have caused problems with the RAID later?

Absolutely. Now we need to figure out how to get from here to an
empty data partition.

After the mount, both of sd[ab]1 would have to be recovered and
usable, but out of sync. So when the raid was force-reassembled, md
would sync the older from the newer. This should, in my book, result
in a valid and full filesystem.

I would assume that the fsck -b run then somehow screwed up the
data. Maybe it's even a bug in fsck -b.

And the other question of course is why the kernel decided it had
any business doing recovery on an fs that was marked for ro mount.

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

[signature.asc (application/pgp-signature, inline)]

Message #45 received at 417407@bugs.debian.org (full text, mbox, reply):

From: martin f krafft <madduck@debian.org>

To: Jim Paris <jim@jtan.com>

Cc: Frans Pop <elendil@planet.nl>, 417407@bugs.debian.org, Peter Nuttall <p.s.nuttall@dur.ac.uk>

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 10:39:26 +0200

[Message part 1 (text/plain, inline)]

also sprach martin f krafft <madduck@debian.org> [2007.04.03.1028 +0200]:
> After the mount, both of sd[ab]1 would have to be recovered and
> usable, but out of sync.

Actually, as Sesse claims, it's entirely likely that md didn't think
the partitions were out of sync. That would explain some pretty bad
data screwage. But an empty filesystem?

-- 
 .''`.   martin f. krafft <madduck@debian.org>
: :'  :  proud Debian developer, author, administrator, and user
`. `'`   http://people.debian.org/~madduck - http://debiansystem.info
  `-  Debian - when you have better things to do than fixing systems

[signature.asc (application/pgp-signature, inline)]

Message #50 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Steve Langasek <vorlon@debian.org>

To: martin f krafft <madduck@debian.org>, 417407@bugs.debian.org

Cc: Jim Paris <jim@jtan.com>, Frans Pop <elendil@planet.nl>, Peter Nuttall <p.s.nuttall@dur.ac.uk>

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 02:11:22 -0700

On Tue, Apr 03, 2007 at 10:39:26AM +0200, martin f krafft wrote:
> also sprach martin f krafft <madduck@debian.org> [2007.04.03.1028 +0200]:
> > After the mount, both of sd[ab]1 would have to be recovered and
> > usable, but out of sync.

> Actually, as Sesse claims, it's entirely likely that md didn't think
> the partitions were out of sync. That would explain some pretty bad
> data screwage. But an empty filesystem?

Checkpoint of the IRC discussion:

- The submitter says that after reboot, the RAID was reported as out of
  sync.
- The logs show that the ext3 filesystem was automatically mounted rw for
  journal recovery by the kernel driver.
- There is no evidence in the logs that the RAID was ever assembled within
  d-i, so it shouldn't be the case that the RAID superblocks were out of
  sync as a result of d-i itself.
- This leaves two possible reasons for the out-of-sync state of the RAID:
  either mounting the individual partitions as ext3 filesystems somehow
  overwrote the RAID superblock just the right way (unlikely since it would
  require the ext3 driver to write past the end of the declared filesystem),
  or the RAID superblocks were out of sync /before/ booting d-i.  The latter
  is consistent with the fact that the ext3 driver had to do a journal
  recovery, suggesting that both the ext3 fs and the RAID were not cleanly
  shut down.
- If mounting as ext3 overwrote the RAID superblock, that seems to be a
  kernel bug, and we have no good explanation for how that would happen.
- If the RAID was unclean before booting d-i, all bets are off as to the
  state of the filesystem at the beginning of this journal recovery, and it
  may be difficult to ever reproduce this bug.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/

Message #55 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Peter Nuttall <p.s.nuttall@durham.ac.uk>

To: 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 10:31:03 +0100

On Tue, Apr 03, 2007 at 08:53:24AM +0200, martin f krafft wrote:
> also sprach Peter Nuttall <p.s.nuttall@durham.ac.uk> [2007.04.03.0020 +0200]:
> > mdadm --assemble /dev/md0 works, as does your command above. The
> > array is fine after resyncing, its just its contents that are
> > gone. 
> 
> I honestly have no idea what could have gone wrong. Can you check
> whether the array's UUID (mdadm -D /dev/md0) was changed or whether
> it's still the same as before the disaster? /etc/mdadm/mdadm.conf
> should list it.
> 

the UUIDs match, but the backup of /etc from before the upgrade is
sitting in /home... So I can't tell you if it matches the pre-etch
UUID. 

Pete

> -- 
>  .''`.   martin f. krafft <madduck@debian.org>
> : :'  :  proud Debian developer, author, administrator, and user
> `. `'`   http://people.debian.org/~madduck - http://debiansystem.info
>   `-  Debian - when you have better things to do than fixing systems

Message #60 received at 417407@bugs.debian.org (full text, mbox, reply):

From: "Steinar H. Gunderson" <sgunderson@bigfoot.com>

To: martin f krafft <madduck@debian.org>, 417407@bugs.debian.org

Cc: Jim Paris <jim@jtan.com>, Frans Pop <elendil@planet.nl>, Peter Nuttall <p.s.nuttall@dur.ac.uk>

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 10:59:48 +0200

On Tue, Apr 03, 2007 at 10:39:26AM +0200, martin f krafft wrote:
>> After the mount, both of sd[ab]1 would have to be recovered and
>> usable, but out of sync.
> Actually, as Sesse claims, it's entirely likely that md didn't think
> the partitions were out of sync. That would explain some pretty bad
> data screwage. But an empty filesystem?

A different scenario might be a write-intent bitmap (--bitmap= to mdadm). Do
you use such a thing?

/* Steinar */
-- 
Homepage: http://www.sesse.net/

Message #65 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Peter Nuttall <p.s.nuttall@durham.ac.uk>

To: 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 11:11:10 +0100

On Tue, Apr 03, 2007 at 10:59:48AM +0200, Steinar H. Gunderson wrote:
> On Tue, Apr 03, 2007 at 10:39:26AM +0200, martin f krafft wrote:
> >> After the mount, both of sd[ab]1 would have to be recovered and
> >> usable, but out of sync.
> > Actually, as Sesse claims, it's entirely likely that md didn't think
> > the partitions were out of sync. That would explain some pretty bad
> > data screwage. But an empty filesystem?
> 
> A different scenario might be a write-intent bitmap (--bitmap= to mdadm). Do
> you use such a thing?

No, 

Pete


> 
> /* Steinar */
> -- 
> Homepage: http://www.sesse.net/
> 

Message #70 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Frans Pop <elendil@planet.nl>

To: Peter Nuttall <p.s.nuttall@dur.ac.uk>, 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 03 Apr 2007 12:25:17 +0200

[Message part 1 (text/plain, inline)]

On Monday 02 April 2007 18:33, Peter Nuttall wrote:
> D-I questions: I just pressed enter at the prompt, and set my keyboard
> and location to enGB. On reaching the disk stage, I asked it to set up
> hda as a LVM with / /home /tmp and /var on seperate paritions. On
> checking the result seemed to suggest no changes to /dev/sda or
> /dev/sdb. I have attached syslog and partman from /var/log/installer.

I have tried to reproduce this issue in vmware, but so far have been 
unable to do so.

- started with clean system with 1 IDE and 2 SCSI disks
- installed Etch with /home on RAID1 as only filesystem on SCSI disks
- copied /usr/share/doc into $HOME

I then rebooted the system cleanly and did another install; this did not 
reproduce the os-prober messages. After that install there were no 
problems: installing mdadm made the RAID available without any problems.

I then did a forced shut down the system so that the file systems would 
not be marked clean. This did reproduce the messages during a new install 
while os-prober was run (same for both partitions):
os-prober: debug: running /usr/lib/os-probes/50mounted-tests 
on /dev/discs/disc1/part1
kernel: EXT2-fs: sda1: couldn't mount because of unsupported optional 
features (4).
kernel: EXT3-fs: INFO: recovery required on readonly filesystem.
kernel: EXT3-fs: write access will be enabled during recovery.
kernel: kjournald starting.  Commit interval 5 seconds
kernel: EXT3-fs: recovery complete.
kernel: EXT3-fs: mounted filesystem with ordered data mode.
50mounted-tests: debug: mounted as ext3 filesystem
[...]
os-prober: debug: running /usr/lib/os-probes/50mounted-tests 
on /dev/discs/disc1/part1
kernel: EXT2-fs warning (device sda1): ext2_fill_super: mounting ext3 
filesystem as ext2
50mounted-tests: debug: mounted as ext2 filesystem

After the reboot I again installed mdadm. However, I could not reproduce 
any problems during this: the RAID was not out of sync, was assembled 
normally and after mounting it all data was there.

My conclusion is that by itself the fact that os-prober mounts the 
partitions that make up a RAID1 does not damage anything, even if they 
were not unmounted cleanly before.

The real question here seems to be: why were the partitions unclean before 
the installation was started.

[Message part 2 (application/pgp-signature, inline)]

Message #75 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Steve Langasek <vorlon@debian.org>

To: Frans Pop <elendil@planet.nl>, 417407@bugs.debian.org

Cc: Peter Nuttall <p.s.nuttall@dur.ac.uk>

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 13:28:31 -0700

On Tue, Apr 03, 2007 at 12:25:17PM +0200, Frans Pop wrote:
> After the reboot I again installed mdadm. However, I could not reproduce 
> any problems during this: the RAID was not out of sync,

In that case, I would say that the test did not accurately model the
submitter's circumstances.

To increase the chances of getting an out-of-sync RAID at this point, could
you try copying a lot of files around in the vm, and *then* killing it in
the middle of that copy operation?  The RAID will after all only be
out-of-sync if there was a write operation in progress that was interrupted
before it was copied to both disks.

> My conclusion is that by itself the fact that os-prober mounts the 
> partitions that make up a RAID1 does not damage anything, even if they 
> were not unmounted cleanly before.

> The real question here seems to be: why were the partitions unclean before 
> the installation was started.

I tend to agree, but I would appreciate seeing results from this one last
test as described above before downgrading this bug...

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/

Message #80 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Samuel Thibault <samuel.thibault@ens-lyon.org>

To: martin f krafft <madduck@debian.org>, 417407@bugs.debian.org, Steve Langasek <vorlon@debian.org>

Cc: Jim Paris <jim@jtan.com>, Frans Pop <elendil@planet.nl>, Peter Nuttall <p.s.nuttall@dur.ac.uk>

Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device

Date: Tue, 3 Apr 2007 19:26:25 +0200

Hi,

martin f krafft, le Tue 03 Apr 2007 10:28:24 +0200, a écrit :
> And the other question of course is why the kernel decided it had
> any business doing recovery on an fs that was marked for ro mount.

Because it always do so, see linux/fs/ext3/super.c:ext3_load_journal():
even if the mount is read-only, the journal is recovered. If (but only
if) the device itself is read-only, then nothing is written back to the
disk. Ext3 clearly lacks xfs' norecovery mount option.

Steve Langasek, le Tue 03 Apr 2007 02:11:22 -0700, a écrit :
> - The logs show that the ext3 filesystem was automatically mounted rw for
>   journal recovery by the kernel driver.

Not mounted rw, but mounted ro with journal recovery.

Samuel

Message #85 received at 417407@bugs.debian.org (full text, mbox, reply):

From: "Alex Owen" <r.alex.owen@gmail.com>

To: 417407@bugs.debian.org

Subject: Re: Bug#417407: debian-installer: possible workarounds for "d-i destroyed existing raid device"

Date: Wed, 4 Apr 2007 08:57:55 +0100

Cut and paste from BTS web interface so sorry for the formatting!

>From: Samuel Thibault <samuel.thibault@ens-lyon.org>
>Subject: Re: Bug#417407: debian-installer: d-i destroyed existing raid device
>Date: Tue, 3 Apr 2007 19:26:25 +0200
>
>Hi,
>
>martin f krafft, le Tue 03 Apr 2007 10:28:24 +0200, a écrit :
>> And the other question of course is why the kernel decided it had
>> any business doing recovery on an fs that was marked for ro mount.
>Because it always do so, see linux/fs/ext3/super.c:ext3_load_journal():
>even if the mount is read-only, the journal is recovered. If (but only
>if) the device itself is read-only, then nothing is written back to the
>disk. Ext3 clearly lacks xfs' norecovery mount option.

I can think of two possible workarounds for this. NB: I have not
looked at the code but I'm just thinking out loud.

[1] at the partitioner stage configure md devices... IIRC this should
recognize the preexisting device. Then mark the device to be mounted
at /home (or /oldhome for the ultra paranoid) and markit to be left
alone (ie not formatted). _Presumably_ os-prober would then ignore the
md device and it's components when looking for other OS's.

[2] wrap the mount/umount sections of code in os-prober with calls to
blockdev to set the block device readonly (and restore old perms on
umount). This would side step the deficiencies in the "unconditional
ext3 journal recovery code". (but this would need a block-dev udeb
added to util-linux source)

Just some thoughts...
Alex Owen

Message #90 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Steve Langasek <vorlon@debian.org>

To: 417407@bugs.debian.org, 417407-submitter@bugs.debian.org

Subject: Re: debian-installer: d-i destroyed existing raid device

Date: Wed, 4 Apr 2007 18:58:35 -0700

severity 417407 important
thanks

At this point, there is a lack of evidence showing that this data loss was
caused by os-prober.  There is, however, evidence that the RAID was in an
inconsistent state before debian-installer ever looked at it -- more
inconsistent than the d-i team was able to forcibly achieve in testing.

Since we no one seems to have any leads on a way forward on this issue, I
don't think it's reasonable to treat it as "critical".  Of course, if
someone does identify a cause for this bug in debian-installer, it would be
reasonable to fix it at the next possible moment.

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
vorlon@debian.org                                   http://www.debian.org/

Message #100 received at 417407@bugs.debian.org (full text, mbox, reply):

From: "Alex Owen" <r.alex.owen@gmail.com>

To: 417407@bugs.debian.org

Subject: Re: Improve safety of os-prober by ignoring active swap and using "blockdev --setro" when available

Date: Sat, 7 Apr 2007 15:27:59 +0100

[Message part 1 (text/plain, inline)]

OK here is the patch!

On 07/04/07, Alex Owen <r.alex.owen@gmail.com> wrote:
> Attached is a patch to os-prober to:
> (1) make it ignore active swap
> [Changes in: os-prober/os-prober]
>
> (2) if blockdev is available then use it to set partitions read-only
> before mounting to work arround a "feature" of ext3 and possible other
> journaled filesystems!
> [Changes in: os-prober/os-probes/common/50mounted-tests]
>

[os-prober.patch (text/x-patch, attachment)]

Message #105 received at 417407@bugs.debian.org (full text, mbox, reply):

From: "Alex Owen" <r.alex.owen@gmail.com>

To: 417407@bugs.debian.org

Subject: Improve safety of os-prober by ignoring active swap and using "blockdev --setro" when available

Date: Sat, 7 Apr 2007 15:26:11 +0100

clone 417407 -1
reassign -1 util-linux
retitle -1 util-linux: wish there was a blockdev-udeb
severity -1 wishlist
reassign 417407 os-prober
tags  417407 +patch
retitle 417407 os-prober: protect partitions with "blockdev --setro"
when available
thanks

Attached is a patch to os-prober to:
(1) make it ignore active swap
[Changes in: os-prober/os-prober]

(2) if blockdev is available then use it to set partitions read-only
before mounting to work arround a "feature" of ext3 and possible other
journaled filesystems!
[Changes in: os-prober/os-probes/common/50mounted-tests]

I do not claim to have tested the patch. I have tried to write the
patch so that (2) does nothing if blockdev is not there... Perhaps
this means the deb should recommend util-linux while the udeb could
require blockdev-udeb?

I have cloned this bug as a wishlist against util-linux requesting a
blockdev-udeb.

Regards
Alex Owen

Message #120 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Frans Pop <elendil@planet.nl>

To: Alex Owen <r.alex.owen@gmail.com>, 417407@bugs.debian.org

Subject: Re: Bug#417407: Improve safety of os-prober by ignoring active swap and using "blockdev --setro" when available

Date: Sat, 07 Apr 2007 17:12:45 +0200

[Message part 1 (text/plain, inline)]

On Saturday 07 April 2007 16:27, Alex Owen wrote:
> OK here is the patch!

We'll have to check if this approach is the correct way to go about this. 
I'm also not yet convinced that what os-prober did was the main cause of 
the data loss.
However, not touching partitions not involved in the installation seems 
like a reasonable precaution in any case, so we'll certainly consider 
this option.

One comment on the patch. I'd suggest to implement protect_dev as follows:
+protect_dev(){ #$1=partition : stdout=dev_rw_flag
+local dev_rw_flag=0
+if type blockdev >/dev/null 2>&1; then
+	if  [ $(blockdev --getro $1) = 0 ] &&
+	    blockdev --setro $1; then 
+		dev_rw_flag=1
+	fi
+fi
+echo $dev_rw_flag

[Message part 2 (application/pgp-signature, inline)]

Message #125 received at 417407@bugs.debian.org (full text, mbox, reply):

From: "Michal Suchanek" <hramrach@centrum.cz>

To: 417407@bugs.debian.org

Subject: installing to a box with suspended system

Date: Wed, 31 Oct 2007 20:00:33 +0200

Hello,

I tried to use the installer on a box which had a Linux system
suspended on one disk. As I noticed that os-prober mounts the
partitions I did not try to resume.
According to the software suspend documentation this could cause
serious data loss, especially since the filesystem loses consistency
while *mounted*.

Thanks

Michal

Message #130 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Frans Pop <elendil@planet.nl>

To: "Michal Suchanek" <hramrach@centrum.cz>

Subject: Re: Bug#417407: installing to a box with suspended system

Date: Wed, 31 Oct 2007 20:04:34 +0100

[Message part 1 (text/plain, inline)]

On Wednesday 31 October 2007, Michal Suchanek wrote:
> I tried to use the installer on a box which had a Linux system
> suspended on one disk. As I noticed that os-prober mounts the
> partitions I did not try to resume.
> According to the software suspend documentation this could cause
> serious data loss, especially since the filesystem loses consistency
> while *mounted*.

The installer will also in principle automatically reuse an existing swap 
partition during the installation...
In my opinion this is a completely different issue than the subject of this 
report.

I also think that trying to do an installation on a system that has an OS in 
suspended state is not very smart. _Any_ installation of an OS is 
inherently risky as you can never be sure exactly what changes it may make 
on disk. Compounding that risk by having operating systems suspended is 
something you should know to avoid as a user.
We could add a warning about that in the installation guide, but I don't 
think there is much more that we can do than that. I would personally 
qualify any data loss resulting from that as being caused by "user error".

Setting follow-up for this to the mailing list as this does not belong in 
this bug report.

Cheers,
FJP

[signature.asc (application/pgp-signature, inline)]

Message #135 received at 417407@bugs.debian.org (full text, mbox, reply):

From: "Eddy Petrișor" <eddy.petrisor@gmail.com>

To: 417407@bugs.debian.org

Subject: Re: Bug#417407: installing to a box with suspended system

Date: Thu, 1 Nov 2007 15:02:33 +0200

On 31/10/2007, Frans Pop <elendil@planet.nl> wrote:
> On Wednesday 31 October 2007, Michal Suchanek wrote:
> > I tried to use the installer on a box which had a Linux system
> > suspended on one disk. As I noticed that os-prober mounts the
> > partitions I did not try to resume.
> > According to the software suspend documentation this could cause
> > serious data loss, especially since the filesystem loses consistency
> > while *mounted*.

> I also think that trying to do an installation on a system that has an OS in
> suspended state is not very smart. _Any_ installation of an OS is
> inherently risky as you can never be sure exactly what changes it may make
> on disk. Compounding that risk by having operating systems suspended is
> something you should know to avoid as a user.
> We could add a warning about that in the installation guide, but I don't
> think there is much more that we can do than that. I would personally
> qualify any data loss resulting from that as being caused by "user error".

I have to agree with Frans on this, by definition a new installation
can even result in data loss and making backups is advised. I would
say that leaving a system in a suspended state fails from the
beginning in the area of "did not do a backup as you were advised".

-- 
Regards,
EddyP
=============================================
"Imagination is more important than knowledge" A.Einstein

Message #140 received at 417407@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: Otavio Salvador <otavio@debian.org>

Cc: debian-boot@lists.debian.org, 418163@bugs.debian.org

Subject: Re: Bug#417407: Improve safety of os-prober by ignoring active swap and using "blockdev --setro" when available

Date: Tue, 9 Nov 2010 10:34:24 +0000

reassign 418163 busybox
thanks

On Sat, Apr 07, 2007 at 12:23:07PM -0300, Otavio Salvador wrote:
> Frans Pop <elendil@planet.nl> writes:
> > Note that despite serious attempts we have not been able to reproduce the 
> > original issue, so it is not even completely certain that a patch is 
> > required at all.
> 
> While I do agree that we're not yet sure that this patch is _required_
> I do support its inclusion since it would make the os-prober _really_ a
> readonly operation avoiding further possible issues.

(Recent d-i team meeting discussion indicates that this is currently
considered a bug we very much want to fix for squeeze.)

blockdev was added to busybox upstream in early September.  I think the
best path forward for this bug is to backport that applet, which should
be straightforward.  I'm working on this now, and will then review
Alex's patch.

-- 
Colin Watson                                       [cjwatson@debian.org]

Message #145 received at 417407-close@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@debian.org>

To: 417407-close@bugs.debian.org

Subject: Bug#417407: fixed in os-prober 1.40

Date: Wed, 10 Nov 2010 12:02:06 +0000

Source: os-prober
Source-Version: 1.40

We believe that the bug you reported is fixed in the latest version of
os-prober, which is due to be installed in the Debian FTP archive:

os-prober-udeb_1.40_i386.udeb
  to main/o/os-prober/os-prober-udeb_1.40_i386.udeb
os-prober_1.40.dsc
  to main/o/os-prober/os-prober_1.40.dsc
os-prober_1.40.tar.gz
  to main/o/os-prober/os-prober_1.40.tar.gz
os-prober_1.40_i386.deb
  to main/o/os-prober/os-prober_1.40_i386.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 417407@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Colin Watson <cjwatson@debian.org> (supplier of updated os-prober package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Wed, 10 Nov 2010 11:51:19 +0000
Source: os-prober
Binary: os-prober-udeb os-prober
Architecture: source i386
Version: 1.40
Distribution: unstable
Urgency: low
Maintainer: Debian Install System Team <debian-boot@lists.debian.org>
Changed-By: Colin Watson <cjwatson@debian.org>
Description: 
 os-prober  - utility to detect other OSes on a set of drives
 os-prober-udeb - utility to detect other OSes on a set of drives (udeb)
Closes: 417407 556739 567953 589676 592924 599203
Changes: 
 os-prober (1.40) unstable; urgency=low
 .
   [ Christian Perrier ]
   * Fix Windows Vista and Windows Recovery Environment partitions
     recognition. (Thanks, Bouke Bunnik)
     Closes: #589676, LP: #476625
   * Allow recognition of recent MINIX installations.
     Thanks to Feiran Zheng
     Closes: #592924
 .
   [ Colin Watson ]
   * Improve error message when /sys/block is missing.
   * os-prober doesn't know how to probe other OSes on non-Linux kernels.
     For now, just exit quietly rather than confusing people (closes:
     #567953).
   * Ignore active swap partitions (thanks, Alex Owen; see #417407).
   * Refactor linux_mount_boot to look up labels and UUIDs using blkid or
     /dev/disk/by-*/ rather than relying on mount being smart enough.  This
     removes some horrible code that executes mount from /target.
   * Set partitions read-only before mounting them (based on a patch by Alex
     Owen; closes: #417407, #556739, #599203).
Checksums-Sha1: 
 045432945afe4bad0671e4ee5a98836e244271e5 1577 os-prober_1.40.dsc
 cdeca296318e51b37233d63424c9b262f41b2db0 23470 os-prober_1.40.tar.gz
 cd9b549f42ad7275579778a1709e93a2568b6904 13156 os-prober-udeb_1.40_i386.udeb
 42649ede8d363bf5c28a5f52febb04c4bd9e966d 23670 os-prober_1.40_i386.deb
Checksums-Sha256: 
 a29c19dd59e82f87d5f6063a0d9e1cc7926339bb0338f08b55c6a6dac625d77b 1577 os-prober_1.40.dsc
 fa4e6dc51521c0f60b5b8157a2bd42475473b958ea93725669a90e17b7cd6a70 23470 os-prober_1.40.tar.gz
 c6ae392745556a2bc8a2d474a83daacc07e3a9829fd3af9588bb9c9bac310b8b 13156 os-prober-udeb_1.40_i386.udeb
 84f4e5b291b94dba9f678fcd8881693a1848ab5ee4ac123c336ad95c2bb1e395 23670 os-prober_1.40_i386.deb
Files: 
 d7a7d3963331e103d9e1c36d65e6aa61 1577 debian-installer optional os-prober_1.40.dsc
 5f441d4962c2be2fa226cc44eee81ab9 23470 debian-installer optional os-prober_1.40.tar.gz
 37ff87a79839ae6cfa9d47bb0f98fd19 13156 debian-installer optional os-prober-udeb_1.40_i386.udeb
 4a82106efb183f2d8929a44bf62f77b2 23670 utils extra os-prober_1.40_i386.deb
Package-Type: udeb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Colin Watson <cjwatson@debian.org> -- Debian developer

iQIVAwUBTNqHqjk1h9l9hlALAQhcHg//U3DmQr6A1PHk8EaR1GUTkkQTcPJ0B/C1
mxi0c8nG3hTVNimmw6W03CWEKjZ6tRzAHxqN0am8LIl5M9BSMgO/tobGnYrNthre
ajr9pNLTrNUihWG7bfD8cra2FjdJAtCB2ZVwqqns8vTKwMEYEutHyTUBYMa8dUed
FKAA9aml86pwnA9Ek1kUtRCHdXlCAe62AMWhTxKCMh3GBY38cALT3xVYegcwrK4R
2sQcJIHpANzGFpNsj0bFJqlCOhxMQzVM3T7yL2+9eLFb4mm7jJdoWRtlhEXvW7Ph
gAnyX5oRIHHNI0W8NSQsSIw71E0+oK7bA420pLfJKrI7myyvlwiPopTljLU3ZHvB
CxkhvELgtv/fOTzaJCb4VsSvi2szS5esvMQD/1DZmeNFjamuybYvNdSdSQaCZtcr
Sb6YVKKM7VD1DrADMugKWaiwWKJvdkBqVuQDFFQr06tNLYMJMauyRZCfEugpxbj2
R22Dgs+A3ooFM7VWTNC/+o3JZjoqYiY6Guis/E1jwKpl4cRnA5uE6DkR+bw4VWUv
v0FiA97CbXULJZZrOKWQcOdXIpLvaLQSmowg4Iu2MaYKtrL5B3jVjo1EkloO3QuY
/i+t8LLL/tJP2YP4n6qz7/MI+JOgs029Sl3fHLLRdP8a/H2MixrXZEWa6qM4gZUQ
00CidGoFO8M=
=WA+X
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.