Debian Bug report logs - #686130
hack for installing with btrfs raid1

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Daniel Pocock <daniel@pocock.com.au>

To: submit@bugs.debian.org

Subject: hack for installing with btrfs raid1

Date: Tue, 28 Aug 2012 23:40:31 +0200

Package: installation-reports

Image version: beta1 DVD amd64
http://cdimage.debian.org/cdimage/wheezy_di_beta1/amd64/iso-dvd/debian-wheezy-DI-b1-amd64-DVD-1.iso
Date: 28 August 2012


Although D-I doesn't have menu-based support for btrfs RAID, I was keen
to find out if there were any other traps later on in the process.

Therefore, I describe the hack I've used to run D-I and install onto
btrfs RAID1

The install is done in expert mode, text interface, using VirtualBox
with two virtual SATA disks of 10GB each.

I successfully proceed through all steps up to partitioning

In the partitioning system, I create raw partitions for LVM on each of
my two disks

sda:
  /dev/sda1  type 83   (ext4 for /boot)
  /dev/sda2   swap
  /dev/sda3  type 8e
sdb:
  /dev/sdb3  type 83

I then do the LVM setup, creating a VG called vg00 with both disks

Next, I leave partman running and press Alt-F2 for a shell

In the shell, I manually create two LVs:

  lvcreate -n lv0 -L 10GB vg00 /dev/sda3
  lvcreate -n lv1 -L 10GB vg00 /dev/sdb3

and I join them into a btrfs RAID1:

  mkfs.btrfs -d raid1 -m raid1 /dev/vg00/lv0 /dev/vg00/lv1

Going back to the D-I menu, I try to `Detect Disks' and back into
partman, it just doesn't recognise the RAID1,
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=686117
   so a further workaround is needed:

I simply ask partman to format /dev/vg00/lv0 alone as btrfs (no RAID)
for mount = / (root filesystem)

Then I am back in the main menu.  Before running the step "Install the
base system", I go back to the shell, (alt-f2)

  umount /target/boot
  umount /target
  mkfs.btrfs -d raid1 -m raid1 /dev/vg00/lv0 /dev/vg00/lv1
  mount /dev/vg00/lv0 /target
  mkdir /target/boot
  mount /dev/sda1 /target/boot

Now I go back to the D-I menu and proceed normally from `Install the
base system'

Note that /etc/fstab is not created correctly.  It is necessary to
manually edit /target/etc/fstab

When it reboots into grub, there is trouble (this may require a fix in
update-initramfs tool or in the initramfs scripts provided by btrfs-tools)

Loading, please wait...
Scanning for Btrfs filesystems
mount: mounting /dev/mapper/vg00-lv0 on /root failed: Invalid argument

and we are dumped in the BusyBox shell.  The first discovery is that
/dev/vg00/lv1 is not active:

(initramfs) lvm
lvm> lvscan
  ACTIVE   /dev/vg00/lv0
  inactive /dev/vg00/lv1
lvm> lvchange -ay vg00
  ACTIVE   /dev/vg00/lv0
  ACTIVE   /dev/vg00/lv1
lvm> exit
(initramfs) btrfs dev scan
Scanning for btrfs filesystems
(initramfs) mount -o ro /dev/vg00/lv0 /root
(initramfs) ls /root

At this point, /root looks normal

However, trying to continue the init (e.g. typing `exit' or trying to
invoke switch_root /root /sbin/init) fails, e.g.


Usage: switch_root [-c /dev/console] NEW_ROOT NEW_INIT [ARGS]


Only the help text appears, no details about why switch_root is unhappy

Message #10 received at 686130@bugs.debian.org (full text, mbox, reply):

From: Daniel Pocock <daniel@pocock.com.au>

To: 686130@bugs.debian.org

Subject: finishing the hack

Date: Wed, 29 Aug 2012 00:05:10 +0200

Some further comments - it finally does boot


Following on from the original install report (stuck in initramfs), I then

a) boot into rescue mode with the Debian beta1 DVD

b) ask the rescue system to mount /dev/vg00/lv0

c) select the option to launch a shell in the root FS

d) hack /usr/share/initramfs-tools/scripts/local-premount/btrfs to make
sure all LVs are active before the btrfs dev scan command (notice I
added the lvchange command below into the existing script):

if [ -x /sbin/btrfs ]
then
        modprobe btrfs
        /sbin/lvm lvchange -ay vg00
        /sbin/btrfs device scan 2> /dev/null
fi


e) regenerate the initramfs

  mount /boot && update-initramfs -k all -u

f) exit the shell and reboot from the rescue mode menu


Now my system boots into a fresh install of wheezy on btrfs RAID1


Action items for full btrfs RAID1 in Debian:

- partman needs to be able to call
    mkfs.btrfs -t raid1 -m raid1
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=686097

- tweak the initramfs support in btrfs-tools (or maybe it is actually an
lvm2 issue?)





root@wheezy3:~# btrfs fil df /
Data, RAID1: total=5.00GB, used=3.18GB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=242.18MB
Metadata: total=8.00MB, used=0.00

root@wheezy3:~# df -lh
Filesystem            Size  Used Avail Use% Mounted on
rootfs                 18G  6.9G  9.5G  42% /
udev                   10M     0   10M   0% /dev
tmpfs                 202M  584K  201M   1% /run
/dev/mapper/vg00-lv0   18G  6.9G  9.5G  42% /
tmpfs                 5.0M     0  5.0M   0% /run/lock
tmpfs                 403M  8.0K  403M   1% /tmp
tmpfs                 403M   72K  403M   1% /run/shm
/dev/sda1             120M   26M   89M  23% /boot

Message #15 received at 686130@bugs.debian.org (full text, mbox, reply):

From: Daniel Pocock <daniel@pocock.com.au>

To: 686130@bugs.debian.org

Subject: related issue in lvm2/initramfs, patch submitted

Date: Wed, 29 Aug 2012 15:10:49 +0200

The patches submitted here:

   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=612402#66

fix the initramfs issue and allow Debian Wheezy to boot from the btrfs RAID1

For this to work, the filesystem must be specified by UUID on the kernel
command line from grub, e.g.


   root=UUID=57f36137-36ec-4f6c-a76e-c72c86330cc2

and should also be specified by UUID in /etc/fstab, e.g.


# cat /etc/fstab
UUID=57f36137-36ec-4f6c-a76e-c72c86330cc2   /   btrfs  defaults 0 1
/dev/sda1 /boot  ext4  defaults 0 2
/dev/sda2  none swap sw 0 0

Message #20 received at 686130@bugs.debian.org (full text, mbox, reply):

From: Daniel Pocock <daniel@pocock.com.au>

To: 686130@bugs.debian.org

Subject: further comments on grub/initramfs issue

Date: Wed, 29 Aug 2012 15:18:43 +0200

initramfs contains the init script.  It accepts one of these syntaxes:

  root=UUID=xxxx
  root=LABEL=xxxx
  root=/dev/mapper/vg-lv   (and other non-LVM devices of course)

If the root=/dev/mapper syntax is used, the lvm2 scripts only lvchange
activate that LV

Whether my patch is accepted or not, there appear to be only three
viable ways to ensure all necessary LVs are active:


a) ALL the logical volumes must be active before the btrfs dev scan
(e.g. using my patch and UUID, or maybe some new config parameter within
lvm.conf that says `activate_all = true')


b) the root= parameter must be extended to include a list of all root
devices that btrfs will need, and all the initramfs logic must be
adapted to handle this new root= parameter


c) some extra config file must be embedded into the initramfs with a
list of the LVs that must be activated

Message #33 received at 686130@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <martin@lichtvoll.de>

To: 686130@bugs.debian.org

Subject: still not booting from BTRFS RAID 1 on two LVs out of the box

Date: Thu, 09 Jun 2016 18:24:31 +0200

Hi!

With todays Debian Sid/Experimental on my laptop this still doesn´t work 
right, I still have to add something like – I just recognized after I 
reinstalled initramfs-tools in a crude way after having messed it up by 
accidentally delete /var/lib/initramfs-tools (and then recognizing that it may 
have been easier to recover it than what I did):

merkaba:/etc/initramfs-tools/scripts/local-top> cat btrfs 
#!/bin/sh

PREREQ="lvm"
prereqs()
{
        echo $PREREQ
}

case $1 in
prereqs)
        prereqs
        exit 0;
esac

. /scripts/functions

log_begin_msg "Initializing BTRFS RAID-1."

modprobe btrfs
vgchange -ay
btrfs device scan

log_end_msg


Despite:

merkaba:/boot> lsinitramfs initrd.img-4.7.0-rc2-tp520-btrfstrim+ | grep 
scripts |  grep btrfs
scripts/local-premount/btrfs
scripts/local-top/btrfs

So whatever local-premount/btrfs does it not sufficient for the BTRFS RAID on 
two LVs case.


I may try again with dracut, but on my last attempt some years ago it also 
didn´t boot this out of the box.

Thanks,
-- 
Martin

Message #38 received at 686130@bugs.debian.org (full text, mbox, reply):

From: Ben Hutchings <ben@decadent.org.uk>

To: Martin Steigerwald <martin@lichtvoll.de>, 686130@bugs.debian.org

Subject: Re: Bug#686130: still not booting from BTRFS RAID 1 on two LVs out of the box

Date: Fri, 10 Jun 2016 00:04:04 +0100

[Message part 1 (text/plain, inline)]

On Thu, 2016-06-09 at 18:24 +0200, Martin Steigerwald wrote:
> Hi!
> 
> With todays Debian Sid/Experimental on my laptop this still doesn´t work 
> right, I still have to add something like – I just recognized after I 
> reinstalled initramfs-tools in a crude way after having messed it up by 
> accidentally delete /var/lib/initramfs-tools (and then recognizing that it may 
> have been easier to recover it than what I did):
[...]
> So whatever local-premount/btrfs does it not sufficient for the BTRFS RAID on 
> two LVs case.

Indeed.  As explained in #565676, lvm2 only activates the VGs
containing the root, /usr and hibernate/resume devices (and only if it
can recognise that they are on LVM).  Apparently it is not generally
safe to activate all visible VGs.  But perhaps that behaviour should be
available as an *option*.  That is safe for most systems and I think it
would resolve all the issues blocking this.

(I don't really understand the purpose of running btrfs on LVM, though.
btrfs integrates logical volume management and I was under the
impression that that's the main reason to use it.  If you still need
another layer underneath then why use it at all?)

Ben.

-- 
Ben Hutchings
Humour is the best antidote to reality.

[signature.asc (application/pgp-signature, inline)]

Message #43 received at 686130@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <martin@lichtvoll.de>

To: Ben Hutchings <ben@decadent.org.uk>

Cc: 686130@bugs.debian.org

Subject: Re: Bug#686130: still not booting from BTRFS RAID 1 on two LVs out of the box

Date: Fri, 10 Jun 2016 10:59:49 +0200

On Freitag, 10. Juni 2016 00:04:04 CEST Ben Hutchings wrote:
> On Thu, 2016-06-09 at 18:24 +0200, Martin Steigerwald wrote:
[…]
> > With todays Debian Sid/Experimental on my laptop this still doesn´t work 
> > right, I still have to add something like – I just recognized after I 
> > reinstalled initramfs-tools in a crude way after having messed it up by 
> > accidentally delete /var/lib/initramfs-tools (and then recognizing that it
> > may 
> > have been easier to recover it than what I did):
> [...]
> 
> > So whatever local-premount/btrfs does it not sufficient for the BTRFS RAID
> > on  two LVs case.
> 
> Indeed.  As explained in #565676, lvm2 only activates the VGs
> containing the root, /usr and hibernate/resume devices (and only if it
> can recognise that they are on LVM).  Apparently it is not generally
> safe to activate all visible VGs.  But perhaps that behaviour should be
> available as an *option*.  That is safe for most systems and I think it
> would resolve all the issues blocking this.

Well, it would definately work on my system.

> (I don't really understand the purpose of running btrfs on LVM, though.
> btrfs integrates logical volume management and I was under the
> impression that that's the main reason to use it.  If you still need
> another layer underneath then why use it at all?)

The reason is simple: I want to have the option to use and test different 
filesystems for my Linux Performance Analysis and Tuning courses and other 
reasons. Also I have two BTRFS RAID 1, the debian system itself aka /, and 
home separately, and one data partition only residing on one SSD.

Once BTRFS allows different RAID levels  in each subvolume it would be 
possible to have this in a single BTRFS, but… I give up the separation of 
operating system and home device thus making it more difficult to reinstall 
the operating system in case it would be needed – I don´t know how good BTRFS 
support is in the installer meanwhile – and probably increasing risk of data 
loss in case of filesystem troubles. Right now when /-device would break, it 
doesn´t affect /home. Having both as a subvolume if one breaks it is more 
likely that it affects the other one as well.

Thanks,
-- 
Martin

Message #48 received at 686130@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <martin@lichtvoll.de>

To: Ben Hutchings <ben@decadent.org.uk>

Cc: 686130@bugs.debian.org

Subject: Re: Bug#686130: still not booting from BTRFS RAID 1 on two LVs out of the box

Date: Sun, 11 Sep 2016 21:56:58 +0200

Am Freitag, 10. Juni 2016, 00:04:04 CEST schrieb Ben Hutchings:
> On Thu, 2016-06-09 at 18:24 +0200, Martin Steigerwald wrote:
> > Hi!
> > 
> > With todays Debian Sid/Experimental on my laptop this still doesn´t work 
> > right, I still have to add something like – I just recognized after I 
> > reinstalled initramfs-tools in a crude way after having messed it up by 
> > accidentally delete /var/lib/initramfs-tools (and then recognizing that it
> > may 
> > have been easier to recover it than what I did):
> [...]
> 
> > So whatever local-premount/btrfs does it not sufficient for the BTRFS RAID
> > on  two LVs case.
> 
> Indeed.  As explained in #565676, lvm2 only activates the VGs
> containing the root, /usr and hibernate/resume devices (and only if it
> can recognise that they are on LVM).  Apparently it is not generally
> safe to activate all visible VGs.  But perhaps that behaviour should be
> available as an *option*.  That is safe for most systems and I think it
> would resolve all the issues blocking this.

Any update on this? I would be fine if it was an option on kernel command line 
or so.

Well I am fine with my initramfs work-around too, but if I ever reinstall 
Debian, which is I admit unlikely as long as I use this ThinkPad T520, that 
would be nice to have.

> Ben Hutchings
> Humour is the best antidote to reality.

Nice one :).

Thanks,
-- 
Martin

Send a report that this bug log contains spam.