Debian Bug report logs - #845526
[PATCH] Allow users to specify the boot directory path

[Message part 1 (text/plain, inline)]
Package: vmdebootstrap
Version: 1.7-1
Severity: wishlist
Tags: patch

Since dpkg-managed files cannot be placed on a FAT32 file system¹, it is
beneficial for platforms such as the Raspberry Pi 3 to mount the FAT32
boot partition of the image in /boot/firmware instead of /boot, and have
maintainer scripts/hooks manage the files in /boot/firmware.

The attached patch adds the --bootdirfmt setting, which can be changed
from its default value %s/boot/ to e.g. %s/boot/firmware/.

Please consider merging the attached patch. Thank you!

① https://wiki.debian.org/Teams/Dpkg/FAQ#Q:_What_are_the_filesystem_requirements_by_dpkg.3F

-- System Information:
Debian Release: stretch/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, armel, mipsel

Kernel: Linux 4.6.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages vmdebootstrap depends on:
ii  debootstrap         1.0.81
ii  kpartx              0.6.2-2
ii  libjs-sphinxdoc     1.4.8-1
ii  parted              3.2-15
ii  python-cliapp       1.20160724-1
ii  python-distro-info  0.14
ii  python2.7           2.7.12-2
pn  python:any          <none>
ii  qemu-utils          1:2.7+dfsg-3+b1

Versions of packages vmdebootstrap recommends:
ii  dosfstools        4.0-2
pn  extlinux          <none>
ii  grub2-common      2.02~beta2-36
pn  python-guestfs    <none>
pn  qemu-system       <none>
ii  qemu-user-static  1:2.7+dfsg-3+b1
pn  squashfs-tools    <none>

Versions of packages vmdebootstrap suggests:
pn  cmdtest       <none>
pn  mbr           <none>
pn  pandoc        <none>
pn  u-boot:armhf  <none>

-- no debconf information

[0001-Allow-users-to-specify-the-boot-directory-path.patch (text/x-diff, attachment)]
[Message part 1 (text/plain, inline)]
On Thu, 24 Nov 2016 10:39:58 +0100 Michael Stapelberg <stapelberg@debian.org> wrote:
> Package: vmdebootstrap
> Version: 1.7-1
> Severity: wishlist
> Tags: patch
> 
> Since dpkg-managed files cannot be placed on a FAT32 file systemÂ¹, it is
> beneficial for platforms such as the Raspberry Pi 3 to mount the FAT32
> boot partition of the image in /boot/firmware instead of /boot, and have
> maintainer scripts/hooks manage the files in /boot/firmware.
> 
> The attached patch adds the --bootdirfmt setting, which can be changed
> from its default value %s/boot/ to e.g. %s/boot/firmware/.
> 
> Please consider merging the attached patch. Thank you!

Why should the argument be a format string?  I can't see why it would
ever be other than "%s" followed by the boot mountpoint, in which case
the "%s" is a redundant bit for users to remember (or forget, screwing
up the image).

Ben.

-- 
Ben Hutchings
Beware of programmers who carry screwdrivers. - Leonard Brandwein


[signature.asc (application/pgp-signature, inline)]
[Message part 1 (text/plain, inline)]
It doesn’t have to be a format string, this was just the least invasive way
that came to my mind. What do you suggest instead?

On Fri, Nov 25, 2016 at 1:42 AM, Ben Hutchings <ben@decadent.org.uk> wrote:

> On Thu, 24 Nov 2016 10:39:58 +0100 Michael Stapelberg <
> stapelberg@debian.org> wrote:
> > Package: vmdebootstrap
> > Version: 1.7-1
> > Severity: wishlist
> > Tags: patch
> >
> > Since dpkg-managed files cannot be placed on a FAT32 file systemÂ¹, it is
> > beneficial for platforms such as the Raspberry Pi 3 to mount the FAT32
> > boot partition of the image in /boot/firmware instead of /boot, and have
> > maintainer scripts/hooks manage the files in /boot/firmware.
> >
> > The attached patch adds the --bootdirfmt setting, which can be changed
> > from its default value %s/boot/ to e.g. %s/boot/firmware/.
> >
> > Please consider merging the attached patch. Thank you!
>
> Why should the argument be a format string?  I can't see why it would
> ever be other than "%s" followed by the boot mountpoint, in which case
> the "%s" is a redundant bit for users to remember (or forget, screwing
> up the image).
>
> Ben.
>
> --
> Ben Hutchings
> Beware of programmers who carry screwdrivers. - Leonard Brandwein
>
>


-- 
Best regards,
Michael

[Message part 2 (text/html, inline)]
[Message part 1 (text/plain, inline)]
the suggested vmdebootstrap patch by Michael doesn't set the customized 
boot mount point in /etc/fstab of the generated images.

i also think, the '--bootdirfmt' and its unusual "%s..." syntax doesn't 
look very handy. therefore i used a simple '--bootdir' option in my fix.

working patch attached.

[0001-corrected-Allow-users-to-specify-the-boot-directory-path.patch (text/x-patch, attachment)]
[Message part 1 (text/plain, inline)]
Hi Martin,

On Sun, Jun 18, 2017 at 11:08 PM, martin schitter <ms+debian@mur.at> wrote:
> the suggested vmdebootstrap patch by Michael doesn't set the customized
boot
> mount point in /etc/fstab of the generated images.
>
> i also think, the '--bootdirfmt' and its unusual "%s..." syntax doesn't
look
> very handy. therefore i used a simple '--bootdir' option in my fix.
>
> working patch attached.

I've tried your patch to build an image, it's simpler and works well.

Regards,

Anisse

[Message part 2 (text/html, inline)]
[Message part 1 (text/plain, inline)]
Hi,

how essential is this? Could it be implemented with a sufficiently
complicated customize script?

I ask because I would like to avoid adding any further features into
vmdebootstrap, which is quite a complicated program already. Instead
all my development work goes into vmdb2 (not yet in Debian), which is
a much simpler program, thanks to a completely different architecture,
and where it would be more feasible to add all sorts of
customizability.

I would like to make vmdebootstrap go into deep maintenance mode,
where I only change things to fix problems that affect official Debian
ISO image building, or when other vmdebootstrap users can't otherwise
build their images and can't switch to vmdb2 either.

On Sun, Jun 18, 2017 at 11:08:07PM +0200, martin schitter wrote:
> the suggested vmdebootstrap patch by Michael doesn't set the customized boot
> mount point in /etc/fstab of the generated images.
> 
> i also think, the '--bootdirfmt' and its unusual "%s..." syntax doesn't look
> very handy. therefore i used a simple '--bootdir' option in my fix.
> 
> working patch attached.

> diff --git a/bin/vmdebootstrap b/bin/vmdebootstrap
> index d9a697d..e4e7e60 100755
> --- a/bin/vmdebootstrap
> +++ b/bin/vmdebootstrap
> @@ -78,6 +78,7 @@ class VmDebootstrap(cliapp.Application):  # pylint: disable=too-many-public-meth
>          self.settings.string(['bootflag'], 'specify flag to set for /boot/', default='')
>          self.settings.bytesize(['bootoffset'], 'Space to leave at start of the '
>                                 'image for bootloader', default='0')
> +        self.settings.string(['bootdir'], 'Mount point of /boot partition', default='/boot/')
>          self.settings.boolean(['use-uefi'], 'Setup image for UEFI boot', default=False)
>          self.settings.bytesize(['esp-size'], 'Size of EFI System Partition - '
>                                 'requires use-uefi', default='5mib')
> @@ -248,9 +249,9 @@ class VmDebootstrap(cliapp.Application):  # pylint: disable=too-many-public-meth
>          elif bootdev:
>              boottype = self.settings['boottype']
>              filesystem.mkfs(bootdev, fstype=boottype)
> -            self.bootdir = '%s/%s' % (rootdir, 'boot/')
> +            self.bootdir = '%s/%s' % (rootdir, self.settings['bootdir'])
>              filesystem.devices['bootdir'] = self.bootdir
> -            os.mkdir(self.bootdir)
> +            os.makedirs(self.bootdir)
>              self.mount(bootdev, self.bootdir)
>  
>          # set user-specified flags, e.g. lba
> diff --git a/doc/overview.rst b/doc/overview.rst
> index 43693f2..da0f236 100644
> --- a/doc/overview.rst
> +++ b/doc/overview.rst
> @@ -107,6 +107,8 @@ Options
>   --boottype=FSTYPE     Filesystem to use for the /boot partition. (default ext2)
>   --bootflag=FLAG       Flag to set on the first partition. (default none)
>   --bootoffset=SIZE     Space to leave at start of the image for bootloader
> + --bootdir=PATH        Mount point of /boot partition.
> +                       Default is ``/boot/``. 
>   --roottype=FSTYPE     Filesystem to use for the / (root) partition. (default ext4)
>   --part-type=PART-TYPE
>                         Partition type to use for this image. (default msdos)
> diff --git a/vmdebootstrap/filesystem.py b/vmdebootstrap/filesystem.py
> index b911c05..d9241c9 100644
> --- a/vmdebootstrap/filesystem.py
> +++ b/vmdebootstrap/filesystem.py
> @@ -171,8 +171,8 @@ class Filesystem(Base):
>              fstab.write('%s / %s %s 0 1\n' %
>                          (rootdevstr, roottype, self.get_mount_flags(roottype)))
>              if bootdevstr:
> -                fstab.write('%s /boot %s %s 0 2\n' %
> -                            (bootdevstr, boottype, self.get_mount_flags(boottype)))
> +                fstab.write('%s %s %s %s 0 2\n' %
> +                            (bootdevstr, self.settings['bootdir'], boottype, self.get_mount_flags(boottype)))
>                  if self.settings['swap'] > 0:
>                      fstab.write("/dev/sda3 swap swap defaults 0 0\n")
>              elif self.settings['swap'] > 0:

> _______________________________________________
> Vmdebootstrap-devel mailing list
> Vmdebootstrap-devel@lists.alioth.debian.org
> https://lists.alioth.debian.org/mailman/listinfo/vmdebootstrap-devel


-- 
I want to build worthwhile things that might last. --joeyh

[signature.asc (application/pgp-signature, inline)]
Hi Lars. :)

[Lars Wirzenius 2017-07-23]
> how essential is this? Could it be implemented with a sufficiently
> complicated customize script?

This patch is one of two for vmdebootstrap to be able to create
Debian Stretch images for Raspberry Pi 3, see
<URL: https://wiki.debian.org/RaspberryPi3 >.  It is key to get a
step further on the path to make Debian an alternative to Raspbian
for RPi owners.

I would love to be able to create Debian images for my RPi without
having to depend on external sources like the
<URL: https://github.com/drtyhlpr/rpi2-gen-image > system. :)

-- 
Happy hacking
Petter Reinholdtsen



[Message part 1 (text/plain, inline)]
I'm afraid I don't think this feature will ever land in vmdebootstrap.
Let me explain.

I initially wrote vmdebootstrap in 2011, and the first version was
very simple and straightforward. However, I quickly ran into
limitations, like not being able to log in as root. To solve this, I
added options such as --root-password. Unfortunately, this was a
mistake. It set a pattern where every time the image built with
vmdebootstrap needed a variation, I added a new option. When Neil took
over as the vmdebootstrap maintainer, he continued this tradition. As
a result, vmdebootstrap is littered with options, many of which are
only really useful for particular users of the program. To test one
build of an image, with one set of options, took me about half an hour
when I took over from Neil. There's at least ten yes/no type options
in vmdebootstap, so a test run that tests just a few of them, say
five, will likely take 2^5 * 0.5 = 16 hours. And that doesn't give
you much test coverage.

Further, this approach has made it difficult to add new features to
vmdebootstrap, and made the code difficult to follow.

To fix this, I've written a vmdebootstrap replacement, vmdb2, which
will be uploaded to Debian later this year. I am reluctant to make any
changes to vmdebootstrap, except to fix bad bugs. I won't be adding
new features.

Sorry.

I hope you will try vmdb2 (when it enters Debian, or see
https://liw.fi/vmdebootstrap/). It's not as featured as vmdebootstrap,
although that's partly by design. If you do try it, please send
feedback on what's good, and what you'd like to see improved. Thanks.

-- 
I want to build worthwhile things that might last. --joeyh

[signature.asc (application/pgp-signature, inline)]
[Lars Wirzenius]
> I'm afraid I don't think this feature will ever land in vmdebootstrap.
> Let me explain.

I sure understand your point of view, and unwillingness to keep working on
vmdeboostrap when you want users to switch to vmdb2, and the problems it
causes for testing the program, but keep in mind that the changes we talk
about here are very small and will have great impact.

The patches in bugs #845526 (52 line diff) and #845439 (30 line diff) will
transform Debian from being a platform _unable_ to create boot images
for one of most popular hobbyist computers (Raspberry Pi), to becoming
a platform where it is possible to create fully working SD card images
for Raspberry Pi 3.  It will open up a huge user segment for Debian that
is currently mostly handled by Raspian, for the many million Raspberry Pi
units sold all over the world.

The changes needed for Raspberry Pi 3 support are very small, and as far
as I can tell make perfect sense:

diff -ur vmdebootstrap-1.8/bin/vmdebootstrap vmdebootstrap-1.8-pere/bin/vmdebootstrap

--- vmdebootstrap-1.8/bin/vmdebootstrap	2017-09-17 16:02:13.000000000 +0200
+++ vmdebootstrap-1.8-pere/bin/vmdebootstrap	2017-09-21 09:25:19.078153178 +0200
@@ -78,6 +78,7 @@
         self.settings.string(['bootflag'], 'specify flag to set for /boot/', default='')
         self.settings.bytesize(['bootoffset'], 'Space to leave at start of the '
                                'image for bootloader', default='0')
+        self.settings.string(['bootdir'], 'Mount point of /boot partition', default='/boot/')
         self.settings.boolean(['use-uefi'], 'Setup image for UEFI boot', default=False)
         self.settings.bytesize(['esp-size'], 'Size of EFI System Partition - '
                                'requires use-uefi', default='5mib')
@@ -248,9 +249,9 @@
         elif bootdev:
             boottype = self.settings['boottype']
             filesystem.mkfs(bootdev, fstype=boottype)
-            self.bootdir = '%s/%s' % (rootdir, 'boot/')
+            self.bootdir = '%s/%s' % (rootdir, self.settings['bootdir'])
             filesystem.devices['bootdir'] = self.bootdir
-            os.mkdir(self.bootdir)
+            os.makedirs(self.bootdir)
             self.mount(bootdev, self.bootdir)
 
         # set user-specified flags, e.g. lba
diff -ur vmdebootstrap-1.8/doc/overview.rst vmdebootstrap-1.8-pere/doc/overview.rst
--- vmdebootstrap-1.8/doc/overview.rst	2017-09-17 16:02:13.000000000 +0200
+++ vmdebootstrap-1.8-pere/doc/overview.rst	2017-09-21 09:25:19.082153224 +0200
@@ -107,6 +107,8 @@
  --boottype=FSTYPE     Filesystem to use for the /boot partition. (default ext2)
  --bootflag=FLAG       Flag to set on the first partition. (default none)
  --bootoffset=SIZE     Space to leave at start of the image for bootloader
+ --bootdir=PATH        Mount point of /boot partition.
+                       Default is ``/boot/``. 
  --roottype=FSTYPE     Filesystem to use for the / (root) partition. (default ext4)
  --part-type=PART-TYPE
                        Partition type to use for this image. (default msdos)
diff -ur vmdebootstrap-1.8/man/overview.rst vmdebootstrap-1.8-pere/man/overview.rst
--- vmdebootstrap-1.8/man/overview.rst	2017-09-17 16:02:13.000000000 +0200
+++ vmdebootstrap-1.8-pere/man/overview.rst	2017-09-21 09:25:19.082153224 +0200
@@ -107,6 +107,8 @@
  --boottype=FSTYPE     Filesystem to use for the /boot partition. (default ext2)
  --bootflag=FLAG       Flag to set on the first partition. (default none)
  --bootoffset=SIZE     Space to leave at start of the image for bootloader
+ --bootdir=PATH        Mount point of /boot partition.
+                       Default is ``/boot/``. 
  --roottype=FSTYPE     Filesystem to use for the / (root) partition. (default ext4)
  --part-type=PART-TYPE
                        Partition type to use for this image. (default msdos)
diff -ur vmdebootstrap-1.8/vmdebootstrap/constants.py vmdebootstrap-1.8-pere/vmdebootstrap/constants.py
--- vmdebootstrap-1.8/vmdebootstrap/constants.py	2017-09-17 16:02:13.000000000 +0200
+++ vmdebootstrap-1.8-pere/vmdebootstrap/constants.py	2017-09-21 09:25:47.482484574 +0200
@@ -46,7 +46,7 @@
         'package': 'grub-efi-arm64',
         'bin_package': 'grub-efi-arm64-bin',
         'extra': None,
-        'exclusive': True,
+        'exclusive': False,
         'target': 'arm64-efi',
     }
 }
diff -ur vmdebootstrap-1.8/vmdebootstrap/filesystem.py vmdebootstrap-1.8-pere/vmdebootstrap/filesystem.py
--- vmdebootstrap-1.8/vmdebootstrap/filesystem.py	2017-09-17 16:02:13.000000000 +0200
+++ vmdebootstrap-1.8-pere/vmdebootstrap/filesystem.py	2017-09-21 09:25:19.086153271 +0200
@@ -171,8 +171,8 @@
             fstab.write('%s / %s %s 0 1\n' %
                         (rootdevstr, roottype, self.get_mount_flags(roottype)))
             if bootdevstr:
-                fstab.write('%s /boot %s %s 0 2\n' %
-                            (bootdevstr, boottype, self.get_mount_flags(boottype)))
+                fstab.write('%s %s %s %s 0 2\n' %
+                            (bootdevstr, self.settings['bootdir'], boottype, self.get_mount_flags(boottype)))
                 if self.settings['swap'] > 0:
                     fstab.write("/dev/sda3 swap swap defaults 0 0\n")
             elif self.settings['swap'] > 0:

Is there something, like testing, we can do to make you less sceptical?

Please reconsider.
-- 
Happy hacking
Petter Reinholdtsen



[Message part 1 (text/plain, inline)]
On Thu, Sep 21, 2017 at 09:37:40AM +0200, Petter Reinholdtsen wrote:
> [Lars Wirzenius]
> > I'm afraid I don't think this feature will ever land in vmdebootstrap.
> > Let me explain.
> 
> I sure understand your point of view, and unwillingness to keep working on
> vmdeboostrap when you want users to switch to vmdb2, and the problems it
> causes for testing the program, but keep in mind that the changes we talk
> about here are very small and will have great impact.

I don't agree, I'm afraid. I understand that Pi users who want Debian
would like vmdebootstrap to support building images for the Pi, but on
the other hand, for me, not having to add features to a code base that
is an unmaintainable mess is more important. I'd prefer Pi users to
help get Pi support in vmdb2 instead.

Also, given that it's my goal to get vmdebootstrap removed from Debian
sooner rather than later, I think it'd be cruel to add a Pi feature
now, and then yank it away in the next instant.

There will be no new features in vmdebootstrap.

-- 
I want to build worthwhile things that might last. --joeyh

[signature.asc (application/pgp-signature, inline)]

Debian Bug report logs - #845526 [PATCH] Allow users to specify the boot directory path

Debian Bug report logs - #845526
[PATCH] Allow users to specify the boot directory path