Debian Bug report logs - #682796
incomplete fai-cd documentation / needless mirror

Package: fai-server; Maintainer for fai-server is Thomas Lange <lange@debian.org>; Source for fai-server is src:fai.

Reported by: "Andreas B. Mundt" <andi.mundt@web.de>

Date: Wed, 25 Jul 2012 19:18:04 UTC

Severity: minor

Done: Thomas Lange <lange@informatik.uni-koeln.de>

Bug is archived. No further changes may be made.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Wed, 25 Jul 2012 19:18:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Andreas B. Mundt" <andi.mundt@web.de>:
New Bug report received and forwarded. Copy sent to Thomas Lange <lange@debian.org>. (Wed, 25 Jul 2012 19:18:06 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: "Andreas B. Mundt" <andi.mundt@web.de>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: fai-server: disk created with fai-cd fails completely if there is no network
Date: Wed, 25 Jul 2012 21:13:53 +0200
Package: fai-server
Severity: important

Hi,

a disk created with fai-cd (4.0.3) tries to access the network to
update the list of available packages instead of using the local
repository on the disk.  From the log:

Calling task_debconf
Adding debconf date from [....

...]
Calling task_repository
[...
fcopy: ... resolv.conf ... /etc/hosts ... sources.list
...]
Calling task_updatebase
Updating base
Err http://security.debian.org stable/updates InRelease
[...]

W: Failed to fetch http://[...]
[...]

Calling task_instsoft
[...]
Couldn't find any package whose name or description matches [...]

If the machine has no network connection, the installation fails
completely because no repository is available at all (the local
mirror is ignored).

Please let me know how to help fixing this for wheezy.

Best regards

     Andi



Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Thu, 26 Jul 2012 09:36:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Andreas B. Mundt" <andi.mundt@web.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Thu, 26 Jul 2012 09:36:06 GMT) Full text and rfc822 format available.

Message #10 received at 682796@bugs.debian.org (full text, mbox):

From: "Andreas B. Mundt" <andi.mundt@web.de>
To: 682796@bugs.debian.org
Subject: suspected commit
Date: Thu, 26 Jul 2012 11:30:27 +0200
Hi,

the commit that breaks installing from a fai-cd seems to be 3b7d917ec8505cea4984a0027:

    https://github.com/faiproject/fai/commit/3b7d917ec8505cea4984a0027fe0348a317cbdb3

+    fcopy -Bv /etc/apt/sources.list || cp -v $FAI_ETC_DIR/apt/sources.list $FAI_ROOT/etc/apt

Is probably not a good idea in case of the cd installation ...

Regards,

	Andi



Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Thu, 26 Jul 2012 10:57:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Andreas B. Mundt" <andi.mundt@web.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Thu, 26 Jul 2012 10:57:03 GMT) Full text and rfc822 format available.

Message #15 received at 682796@bugs.debian.org (full text, mbox):

From: "Andreas B. Mundt" <andi.mundt@web.de>
To: 682796@bugs.debian.org
Cc: control@bugs.debian.org
Subject: patch: use sources.list from CD
Date: Thu, 26 Jul 2012 12:55:08 +0200
tags 682796 patch
thanks

Hi,

please find below a patch fixing the problem.  There might be better
ways to determine that we are installing from a CD image.

Best regards,

     Andi


From b572da44634e2bb32579e0d54be07dd9b6ff7ff0 Mon Sep 17 00:00:00 2001
From: "Andreas B. Mundt" <andi.mundt@web.de>
Date: Thu, 26 Jul 2012 12:41:40 +0200
Subject: [PATCH] Use the local mirror from the cd when installing from CD.

---
 lib/subroutines |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/lib/subroutines b/lib/subroutines
index e1da83f..7c52bd8 100644
--- a/lib/subroutines
+++ b/lib/subroutines
@@ -952,6 +952,11 @@ task_repository () {
     fi
     fcopy -ir /etc/apt # copy all other apt config files from the config space

+    if [ -f $FAI_ETC_DIR/apt/sources.list ] && grep -q cskoeln $FAI_ETC_DIR/apt/sources.list ; then
+	echo "We install from a fai-cd image; use mirror from CD:"
+	cp -vr $FAI_ETC_DIR/apt/* $FAI_ROOT/etc/apt/
+    fi
+
     if [ X$FAI_ALLOW_UNSIGNED = X1 ]; then
         cat <<EOF > $FAI_ROOT/etc/apt/apt.conf.d/10fai
 APT::Get::AllowUnauthenticated "true";
--
1.7.10.4





--

----------------------------------

A N D R E A S   B.   M U N D T

GPG key: 4096R/617B586D 2010-03-22 Andreas B. Mundt--<andreas.b.mundt@web.de>
                                   Andreas B. Mundt--<andi.mundt@web.de>

============================================================================



Added tag(s) patch. Request was from "Andreas B. Mundt" <andi.mundt@web.de> to control@bugs.debian.org. (Thu, 26 Jul 2012 10:57:07 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Sat, 10 Nov 2012 20:06:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Thomas Lange <lange@informatik.uni-koeln.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Sat, 10 Nov 2012 20:06:03 GMT) Full text and rfc822 format available.

Message #22 received at 682796@bugs.debian.org (full text, mbox):

From: Thomas Lange <lange@informatik.uni-koeln.de>
To: 682796@bugs.debian.org, 682796-submitter@bugs.debian.org
Subject: Improved patch
Date: Sat, 10 Nov 2012 21:03:14 +0100
This is not a proper way to check if it's an installation from CD:
   grep -q cskoeln $FAI_ETC_DIR/apt/sources.list

It's better to check if /etc/RUNNING_FROM_FAICD exists.

-- 
regards Thomas



Message sent on to "Andreas B. Mundt" <andi.mundt@web.de>:
Bug#682796. (Sat, 10 Nov 2012 20:06:04 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Sun, 11 Nov 2012 08:45:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Andreas B. Mundt" <andi.mundt@web.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Sun, 11 Nov 2012 08:45:05 GMT) Full text and rfc822 format available.

Message #30 received at 682796@bugs.debian.org (full text, mbox):

From: "Andreas B. Mundt" <andi.mundt@web.de>
To: 682796@bugs.debian.org
Subject: Re: Bug#682796: Improved patch
Date: Sun, 11 Nov 2012 09:41:01 +0100
Hi,

great to see some progress on this issue.

On Sat, Nov 10, 2012 at 09:03:14PM +0100, Thomas Lange wrote:
> 
> This is not a proper way to check if it's an installation from CD:
>    grep -q cskoeln $FAI_ETC_DIR/apt/sources.list
> 
> It's better to check if /etc/RUNNING_FROM_FAICD exists.


Reworked patch just tested here:

diff --git a/lib/subroutines b/lib/subroutines
index e1da83f..32651f7 100644
--- a/lib/subroutines
+++ b/lib/subroutines
@@ -952,6 +952,11 @@ task_repository () {
     fi
     fcopy -ir /etc/apt # copy all other apt config files from the config space
 
+    if [ -f $FAI_ETC_DIR/apt/sources.list ] && [ -e /etc/RUNNING_FROM_FAICD ] ; then
+       echo "We install from a fai-cd image; use mirror from CD:"
+       cp -vr $FAI_ETC_DIR/apt/* $FAI_ROOT/etc/apt/
+    fi
+
     if [ X$FAI_ALLOW_UNSIGNED = X1 ]; then
         cat <<EOF > $FAI_ROOT/etc/apt/apt.conf.d/10fai
 APT::Get::AllowUnauthenticated "true";

 
Best regards,

	Andi



Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Thu, 15 Nov 2012 14:00:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Thomas Lange <lange@informatik.uni-koeln.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Thu, 15 Nov 2012 14:00:03 GMT) Full text and rfc822 format available.

Message #35 received at 682796@bugs.debian.org (full text, mbox):

From: Thomas Lange <lange@informatik.uni-koeln.de>
To: 682796@bugs.debian.org, 682796-submitter@bugs.debian.org
Subject: cannot reproduce this bug
Date: Thu, 15 Nov 2012 14:58:12 +0100
I cannot reproduce this bug using the simple examples shiped with FAI.

The problem seems that your config space contains a template in
files/etc/apt/sources.list/ which is copied via fcopy during
task_repository. The default config space does not contain a template
in sources.list and therefore uses the source.list which only refers
the package repository on the CD.

It's more a general problem. FAI is using fcopy for shipping files
during the installation. But sometimes, you need a file (for
e.g. sources.list) in different versions. One version is used during
installation, another version is used after the new installed system
is running. In case of an installation from CD, this might be a
sources.list which only uses the packages from the CD (during
installation) but the system is configured to use a network repository
after the installation.

fcopy cannot handle those different cases by using the same list of
classes. I also do not like to implement the function you have
proposed (different behaviour depending on installation from network
or CD). 

If you like to ship different sources.list file, you may want to use a
hook, which calls fcopy /etc/apt/sources.list with a different list of
classes. Then you can ship different templates for sources.list during
installation and for configuring the new system.

-- 
regards Thomas



Message sent on to "Andreas B. Mundt" <andi.mundt@web.de>:
Bug#682796. (Thu, 15 Nov 2012 14:00:05 GMT) Full text and rfc822 format available.

Severity set to 'normal' from 'important' Request was from Thomas Lange <lange@debian.org> to control@bugs.debian.org. (Thu, 15 Nov 2012 14:51:04 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Thu, 15 Nov 2012 19:57:06 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Andreas B. Mundt" <andi.mundt@web.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Thu, 15 Nov 2012 19:57:06 GMT) Full text and rfc822 format available.

Message #45 received at 682796@bugs.debian.org (full text, mbox):

From: "Andreas B. Mundt" <andi.mundt@web.de>
To: 682796@bugs.debian.org
Subject: Re: Bug#682796: cannot reproduce this bug
Date: Thu, 15 Nov 2012 20:52:06 +0100
Hi Thomas,

first, thank you for looking into this issue.

On Thu, Nov 15, 2012 at 02:58:12PM +0100, Thomas Lange wrote:
> I cannot reproduce this bug using the simple examples shiped with FAI.
>
> The problem seems that your config space contains a template in
> files/etc/apt/sources.list/ which is copied via fcopy during
> task_repository. The default config space does not contain a template
> in sources.list and therefore uses the source.list which only refers
> the package repository on the CD.

Indeed, my config space distributes a sources.list file.

> It's more a general problem. FAI is using fcopy for shipping files
> during the installation. But sometimes, you need a file (for
> e.g. sources.list) in different versions. One version is used during
> installation, another version is used after the new installed system
> is running. In case of an installation from CD, this might be a
> sources.list which only uses the packages from the CD (during
> installation) but the system is configured to use a network repository
> after the installation.
>

I think the problem is that this behaviour changed with commit

https://github.com/faiproject/fai/commit/3b7d917ec8505cea4984a0027fe0348a317cbdb3

Before, the CD installation used its mirror on the CD, the fcopy of
the sources.list from the config space happened later, in the configure
task [1].

> fcopy cannot handle those different cases by using the same list of
> classes. I also do not like to implement the function you have
> proposed (different behaviour depending on installation from network
> or CD).
>

I guess the way it is in 4.0.3, CD installations use only the CD
mirror if there is no sources.list provided in the config space.  This
is fine if you want and need no other sources later, which is probably
a rare case today.

If you provide a sources.list, it will be used already for the
installation.  This might be fine too (if you know about it and
provide the network access).  It's perhaps even a nice feature,
because you are not limited to CD storage volume/mirror size.

However, in that case the mirror on the CD is useless and should be
skipped completely.  I hacked fai-cd to produce such a CD image
already for the fai version provided in squeeze, it's pretty simple.

> If you like to ship different sources.list file, you may want to use a
> hook, which calls fcopy /etc/apt/sources.list with a different list of
> classes. Then you can ship different templates for sources.list during
> installation and for configuring the new system.
>

I suggest something like the following:

  * Provide an option in fai-cd to not use a CD mirror at all.
  * If the mirror exists, use it for the installation and fcopy the
    sources list later.

What do you think?  I would volunteer to provide a patch for that
scenario if you are interested, but maybe I am not aware of all
problems that might pop up.

Best regards,

     Andi


[1] FAI manual
http://fai-project.org/fai-guide/_anchor_id_config_xreflabel_config_installation_details.html

configure
    Calls scripts in $FAI/scripts/ and its subdirectories for every
    defined class.



Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Thu, 15 Nov 2012 20:24:05 GMT) Full text and rfc822 format available.

Acknowledgement sent to Thomas Lange <lange@informatik.uni-koeln.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Thu, 15 Nov 2012 20:24:05 GMT) Full text and rfc822 format available.

Message #50 received at 682796@bugs.debian.org (full text, mbox):

From: Thomas Lange <lange@informatik.uni-koeln.de>
To: "Andreas B. Mundt" <andi.mundt@web.de>, 682796@bugs.debian.org
Subject: Re: Bug#682796: cannot reproduce this bug
Date: Thu, 15 Nov 2012 21:12:10 +0100
>>>>> On Thu, 15 Nov 2012 20:52:06 +0100, "Andreas B. Mundt" <andi.mundt@web.de> said:

    > I suggest something like the following:

    >   * Provide an option in fai-cd to not use a CD mirror at all.
    >   * If the mirror exists, use it for the installation and fcopy the
    >     sources list later.

    > What do you think?  I would volunteer to provide a patch for that
    > scenario if you are interested, but maybe I am not aware of all
    > problems that might pop up.
I do not like to have a special function inside the FAI CD. The FAI CD
should use the config space to do whatever is defined there. If you
need a special function, use the config space for that, but do not
expect the FAI code to handle such special cases.

FAI is the general, flexible tool. Your own config space is for your
local special needs. I therefore like to close the bug.

-- 
regards Thomas



Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Fri, 16 Nov 2012 14:09:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Andreas B. Mundt" <andi.mundt@web.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Fri, 16 Nov 2012 14:09:03 GMT) Full text and rfc822 format available.

Message #55 received at 682796@bugs.debian.org (full text, mbox):

From: "Andreas B. Mundt" <andi.mundt@web.de>
To: 682796@bugs.debian.org
Cc: control@bugs.debian.org
Subject: Re: Bug#682796: cannot reproduce this bug
Date: Fri, 16 Nov 2012 15:07:50 +0100
retitle 682796 incomplete fai-cd documentation / needless mirror
severity 682796 minor
tags 682796 - patch
thanks


Hi Thomas,

On Thu, Nov 15, 2012 at 09:12:10PM +0100, Thomas Lange wrote:
> >>>>> On Thu, 15 Nov 2012 20:52:06 +0100, "Andreas B. Mundt" <andi.mundt@web.de> said:
>
>     > I suggest something like the following:
>
>     >   * Provide an option in fai-cd to not use a CD mirror at all.
>     >   * If the mirror exists, use it for the installation and fcopy the
>     >     sources list later.
>
>     > What do you think?  I would volunteer to provide a patch for that
>     > scenario if you are interested, but maybe I am not aware of all
>     > problems that might pop up.
> I do not like to have a special function inside the FAI CD. The FAI CD
> should use the config space to do whatever is defined there. If you
> need a special function, use the config space for that, but do not
> expect the FAI code to handle such special cases.
>
> FAI is the general, flexible tool. Your own config space is for your
> local special needs. I therefore like to close the bug.

OK, that's fine.  However, the documentation should be updated in that
case in my opinion.

It should be clear, that the mirror on the CD is only used when no
other APT sources are available in the config space in files/etc/apt/.

Correct me if I am wrong, but it looks as currently a sources.list in
files/etc/apt/ will be used, even if it is not fcopied to the target
in a config space script later.  Usually, the use of a file in files/
needs to be explicitly fcopied (and this happens in a later task).
The exception here might confuse users (at least it confused me).

If the sources.list is provided, the mirror on the CD is ignored.
However, it has to be available when creating the CD (fai-cd man page:
-m is mandatory).  If an empty directory is used in
   fai-cd -m $EMPTYDIR fai-cd.iso
an error is reported:  "No mirror found in $EMPTYDIR. Empty directory."

So I have to prepare a (dummy) repository which will never be used.
Perhaps it's possible to make this error a warning like "No mirror
will be available on the CD; an accessible package repository needs to
be defined in the config space and available during installation."
In addition, allow for missing (or empty) repositories.

So to sum up: I see two shortcomings that should be addressed in code
modifications and/or documentation:

  1) The cases when a mirror on the CD is going to be used is not
     documented.  The user expects that the mirror is used in any
     case, because its provision when creating the CD is mandatory.

  2) A mirror is needed to prepare a FAI CD even if it will never be
     used.

Best regards,

     Andi




Changed Bug title to 'incomplete fai-cd documentation / needless mirror' from 'fai-server: disk created with fai-cd fails completely if there is no network' Request was from "Andreas B. Mundt" <andi.mundt@web.de> to control@bugs.debian.org. (Fri, 16 Nov 2012 14:09:10 GMT) Full text and rfc822 format available.

Severity set to 'minor' from 'normal' Request was from "Andreas B. Mundt" <andi.mundt@web.de> to control@bugs.debian.org. (Fri, 16 Nov 2012 14:09:11 GMT) Full text and rfc822 format available.

Removed tag(s) patch. Request was from "Andreas B. Mundt" <andi.mundt@web.de> to control@bugs.debian.org. (Fri, 16 Nov 2012 14:09:11 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Sat, 17 Nov 2012 18:09:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to Thomas Lange <lange@informatik.uni-koeln.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Sat, 17 Nov 2012 18:09:03 GMT) Full text and rfc822 format available.

Message #66 received at 682796@bugs.debian.org (full text, mbox):

From: Thomas Lange <lange@informatik.uni-koeln.de>
To: "Andreas B. Mundt" <andi.mundt@web.de>, 682796@bugs.debian.org
Subject: Re: Bug#682796: cannot reproduce this bug
Date: Sat, 17 Nov 2012 19:06:35 +0100
>>>>> On Fri, 16 Nov 2012 15:07:50 +0100, "Andreas B. Mundt" <andi.mundt@web.de> said:

    > retitle 682796 incomplete fai-cd documentation / needless mirror
    > severity 682796 minor
    > tags 682796 - patch
Instead of retitling and changing the severity again, I prefer closing
this bug and maybe opening a new one with the new title.



Information forwarded to debian-bugs-dist@lists.debian.org, Thomas Lange <lange@debian.org>:
Bug#682796; Package fai-server. (Sun, 18 Nov 2012 09:06:03 GMT) Full text and rfc822 format available.

Acknowledgement sent to "Andreas B. Mundt" <andi.mundt@web.de>:
Extra info received and forwarded to list. Copy sent to Thomas Lange <lange@debian.org>. (Sun, 18 Nov 2012 09:06:03 GMT) Full text and rfc822 format available.

Message #71 received at 682796@bugs.debian.org (full text, mbox):

From: "Andreas B. Mundt" <andi.mundt@web.de>
To: 682796@bugs.debian.org
Subject: Re: Bug#682796: cannot reproduce this bug
Date: Sun, 18 Nov 2012 10:02:34 +0100
Hi Thomas,

On Sat, Nov 17, 2012 at 07:06:35PM +0100, Thomas Lange wrote:
> >>>>> On Fri, 16 Nov 2012 15:07:50 +0100, "Andreas B. Mundt" <andi.mundt@web.de> said:
> 
>     > retitle 682796 incomplete fai-cd documentation / needless mirror
>     > severity 682796 minor
>     > tags 682796 - patch
> Instead of retitling and changing the severity again, I prefer closing
> this bug and maybe opening a new one with the new title.

No problem for me, feel free to do so.

Regards,

	Andi



Reply sent to Thomas Lange <lange@informatik.uni-koeln.de>:
You have taken responsibility. (Sun, 18 Nov 2012 18:15:13 GMT) Full text and rfc822 format available.

Notification sent to "Andreas B. Mundt" <andi.mundt@web.de>:
Bug acknowledged by developer. (Sun, 18 Nov 2012 18:15:13 GMT) Full text and rfc822 format available.

Message #76 received at 682796-done@bugs.debian.org (full text, mbox):

From: Thomas Lange <lange@informatik.uni-koeln.de>
To: 682796-done@bugs.debian.org
Date: Sun, 18 Nov 2012 19:12:42 +0100
Closing this bug as discussed.
-- 
regards Thomas



Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Mon, 17 Dec 2012 07:28:51 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Thu Apr 24 21:47:00 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.