Debian Bug report logs - #464118
rm -r broken: Function not implemented (using coreutils 6.10 with a pre-etch kernel)

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Marc Leeman <marc.leeman@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: rm -r broken: Function not implemented

Date: Tue, 05 Feb 2008 10:01:40 +0100

Package: coreutils
Version: 6.10-3
Severity: grave


I installed this package, after which my entire system started to break
due to problems caused in mkinitramfs and packages fail to install.

/var/cache/apt/archives/coreutils_6.10-3_i386.deb

After using a boot CD, chrooting and re-installing 
/root/coreutils_5.97-5.3_i386.deb

rm -r was working as expected again.


-- System Information:
Debian Release: lenny/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (700, 'experimental'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 2.6.22.1 (PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages coreutils depends on:
ii  libacl1                      2.2.45-1    Access control list shared library
ii  libc6                        2.7-6       GNU C Library: Shared libraries
ii  libselinux1                  2.0.15-2+b1 SELinux shared libraries

coreutils recommends no packages.

-- no debconf information

Message #10 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Lucas Nussbaum <lucas@lucas-nussbaum.net>

To: marc.leeman@gmail.com, 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Tue, 5 Feb 2008 10:32:11 +0100

On 05/02/08 at 10:01 +0100, Marc Leeman wrote:
> Package: coreutils
> Version: 6.10-3
> Severity: grave
> 
> 
> I installed this package, after which my entire system started to break
> due to problems caused in mkinitramfs and packages fail to install.
> 
> /var/cache/apt/archives/coreutils_6.10-3_i386.deb
> 
> After using a boot CD, chrooting and re-installing 
> /root/coreutils_5.97-5.3_i386.deb
> 
> rm -r was working as expected again.

Hi Marc,

rm -r works fine here:
*** lucas@beothuk:/tmp/cu$ mkdir -p t/{a,b}/{a,b}
*** lucas@beothuk:/tmp/cu$ touch t/{a,b}/{a,b}/f
*** lucas@beothuk:/tmp/cu$ find t
t
t/a
t/a/a
t/a/a/f
t/a/b
t/a/b/f
t/b
t/b/a
t/b/a/f
t/b/b
t/b/b/f
*** lucas@beothuk:/tmp/cu$ rm -r t
*** lucas@beothuk:/tmp/cu$ echo $?
0

and update-initramfs works fine as well:
*** lucas@beothuk:/tmp/cu$ sudo update-initramfs -k 2.6.23-1-686 -u
update-initramfs: Generating /boot/initrd.img-2.6.23-1-686
*** lucas@beothuk:/tmp/cu$ echo $?
0

Could you point to something specific that is/was broken for you?

The output of strace rm -r ... could be useful too.
-- 
| Lucas Nussbaum
| lucas@lucas-nussbaum.net   http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr             GPG: 1024D/023B3F4F |

Message #15 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Lucas Nussbaum <lucas@lucas-nussbaum.net>

To: marc.leeman@gmail.com, 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Tue, 5 Feb 2008 11:35:39 +0100

On 05/02/08 at 10:01 +0100, Marc Leeman wrote:
> rm -r was working as expected again.

On second thought, it might be the same problem as the mips build
failure (the one seen on the buildds). Which filesystem are you using ?
-- 
| Lucas Nussbaum
| lucas@lucas-nussbaum.net   http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr             GPG: 1024D/023B3F4F |

Message #20 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: marc.leeman@gmail.com, 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Tue, 05 Feb 2008 06:24:20 -0500

On Tue, Feb 05, 2008 at 10:01:40AM +0100, Marc Leeman wrote:
>I installed this package, after which my entire system started to break
>due to problems caused in mkinitramfs and packages fail to install.

I need a lot more information that this; it's certainly the case that it 
works fine here & on other people's systems (so can't be duplicated). Do 
you have any sample output? Lucas Nussbaum asked some other questions 
in another email. You can do something like 
 mkdir /tmp/t
 dpkg-deb -x coreutils_6.10-3_i386.deb /tmp/t
 /tmp/t/bin/rm 
to run the 6.10 binary without installing the 6.10 package.

Mike Stone

Message #25 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Adam Conrad <adconrad@0c3.net>

To: 464118@bugs.debian.org

Subject: re: rm -r broken: Function not implemented

Date: Fri, 15 Feb 2008 15:47:34 -0700

For what it's worth, the confusion in this bug log seems due to the
fact that people testing succesfully clearly have /proc mounted.

I spotted the same bug on the Ubuntu buildds today while mucking
around inside chroots without /proc mounted:

https://bugs.edge.launchpad.net/ubuntu/+source/coreutils/+bug/192239

... Adam

Message #30 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: Adam Conrad <adconrad@0c3.net>, 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Fri, 15 Feb 2008 17:56:15 -0500

On Fri, Feb 15, 2008 at 03:47:34PM -0700, you wrote:
>For what it's worth, the confusion in this bug log seems due to the
>fact that people testing succesfully clearly have /proc mounted.
>
>I spotted the same bug on the Ubuntu buildds today while mucking
>around inside chroots without /proc mounted:
>
>https://bugs.edge.launchpad.net/ubuntu/+source/coreutils/+bug/192239

It should certainly be easier to fix with a reproducable test case!

Mike Stone

Message #35 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: Adam Conrad <adconrad@0c3.net>, 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Fri, 15 Feb 2008 18:04:01 -0500

On Fri, Feb 15, 2008 at 05:56:14PM -0500, you wrote:
>On Fri, Feb 15, 2008 at 03:47:34PM -0700, you wrote:
>>For what it's worth, the confusion in this bug log seems due to the
>>fact that people testing succesfully clearly have /proc mounted.
>>
>>I spotted the same bug on the Ubuntu buildds today while mucking
>>around inside chroots without /proc mounted:
>>
>>https://bugs.edge.launchpad.net/ubuntu/+source/coreutils/+bug/192239
>
>It should certainly be easier to fix with a reproducable test case!

of course, I still can't reproduce it:

annuminas:/tmp# find /proc
/proc
annuminas:/tmp# mkdir -p foo/bar/baz
annuminas:/tmp# touch foo/bar/baz/ungh
annuminas:/tmp# rm -rf foo/
annuminas:/tmp# find foo
find: foo: No such file or directory
annuminas:/tmp# 

Am I missing something?

Mike Stone

Message #40 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Adam Conrad <adconrad@0c3.net>

To: Michael Stone <mstone@debian.org>

Cc: 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Fri, 15 Feb 2008 16:23:33 -0700

On Fri, Feb 15, 2008 at 06:04:01PM -0500, Michael Stone wrote:
> 
> of course, I still can't reproduce it:
> 
> Am I missing something?

Damn.  I can tell you that I reproduced it on amd64, i386, powerpc, ia64,
and sparc, and that it was a bare hardy --variant=buildd chroot (more or
less) without /proc mounted.

There's nothing proprietary about our buildd chroots, so I could make a
tarball available to you if that would help.

... Adam

Message #45 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Adam Conrad <adconrad@0c3.net>

To: 464118@bugs.debian.org

Subject: rm -r broken: Function not implemented

Date: Fri, 15 Feb 2008 17:09:31 -0700

What we were missing here was kernel build and/or version, it seems.

I vaguely remembered *not* seeing this on our hppa buildds and, on a hunch,
went checking kernel versions.  amd64, i386, powerpc, ia64, and sparc were
all running 2.6.15, lpia was running 2.6.17, and hppa is running the latest
2.6.22-14-hppa32 from gutsy.

On a hunch, I grabbed a buildd tarball and tried to reproduce this on my
laptop (also running 2.6.22-14 from gutsy, but on i386), and couldn't
reproduce it here locally either.

So, this seems to be restricted to either older kernels, or kernels with
a specific config.

... Adam

Message #50 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Adam Conrad <adconrad@0c3.net>

To: Michael Stone <mstone@debian.org>

Cc: 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Fri, 15 Feb 2008 18:02:02 -0700

[Message part 1 (text/plain, inline)]

On Fri, Feb 15, 2008 at 07:16:12PM -0500, Michael Stone wrote:
> Could you send an strace from one of the non-working systems? That might 
> be enough to figure out what's going wrong and whether it can be worked 
> around.

Attached.

... Adam

[rm.trace (text/plain, attachment)]

Message #55 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Jim Meyering <jim@meyering.net>

To: Adam Conrad <adconrad@0c3.net>

Cc: 464118@bugs.debian.org, Michael Stone <mstone@debian.org>

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Sat, 16 Feb 2008 09:48:44 +0100

Adam Conrad <adconrad@0c3.net> wrote:
> On Fri, Feb 15, 2008 at 07:16:12PM -0500, Michael Stone wrote:
>> Could you send an strace from one of the non-working systems? That might
>> be enough to figure out what's going wrong and whether it can be worked
>> around.
>
> Attached.

Thanks.

Please double-check the version of the rm binary you
used to create that strace output.
When I try to reproduce the situation (no openat support
and no /proc), the fchdir-based openat emulation works
just fine.  But I'm merely simulating the situation,
and am actually running tests on a system with a recent
kernel, so there may be other factors.

That said, ...
I used a version of rm from the trunk (i.e., post-coreutils-6.10)
and comparing an strace of my working rm -r dir/ command to yours
shows fundamental differences that make me think your version of
rm is out of date.   Maybe even from coreutils-5.9x.

If you can confirm you're using something based on 6.10, please
also attach config.status and lib/config.h.

-----------------------
For the record, here's what I did:

Simulate the lack of openat functions:
    ac_cv_func_openat=no ./configure && make && make check
All tests passed.

Next, pretend we don't have /proc/self/fd support either, by changing
    the openat-emulation code to use nonexistent /proc/self/FD:
  perl -pi -e 's,/proc/self/fd,/proc/self/FD,' lib/openat-proc.c \
    tests/du/long-from-unreadable tests/rm/inaccessible

That passed all tests, too, and gave the strace results that looked
so different from yours.

Message #60 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: Jim Meyering <jim@meyering.net>, 464118@bugs.debian.org

Cc: Adam Conrad <adconrad@0c3.net>

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Sat, 16 Feb 2008 07:18:35 -0500

[Message part 1 (text/plain, inline)]

My thought is that glibc has some kind of fallback code if you use 
unlinkat on a system that doesn't support it, and that's what's broken; 
since these are all being compiled with the same glibc, the libc symbol 
should always exist, no? Adam, could you try building/running the 
attached? If it also fails, it ain't coreutils.

Mike Stone

[test.c (text/x-csrc, attachment)]

Message #65 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Jim Meyering <jim@meyering.net>

To: 464118@bugs.debian.org

Cc: Adam Conrad <adconrad@0c3.net>, Michael Stone <mstone@debian.org>

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Sat, 16 Feb 2008 14:53:31 +0100

Jim Meyering <jim@meyering.net> wrote:
> For the record, here's what I did:
>
> Simulate the lack of openat functions:
>     ac_cv_func_openat=no ./configure && make && make check
> All tests passed.
>
> Next, pretend we don't have /proc/self/fd support either, by changing
>     the openat-emulation code to use nonexistent /proc/self/FD:
>   perl -pi -e 's,/proc/self/fd,/proc/self/FD,' lib/openat-proc.c \
>     tests/du/long-from-unreadable tests/rm/inaccessible
>
> That passed all tests, too, and gave the strace results that looked
> so different from yours.

Actually, Michael pointed out that your strace includes an fstatat call
(failing with ENOSYS, no less), which suggests you're using a new-enough
version.

I dug a little deeper and think I have identified the problem.  I suspect
that your system has a working openat function and that coreutils detects
it, but that your fstatat function always returns ENOSYS.

The openat module's configure-time test checks only for the existence
of the openat function.  If that test succeeds on your system, yet it
lacks fstatat, then that would explain what's happening.

To support such a system you have a choice:

  1) easy: turn off all openat support. i.e., build like this:

    ac_cv_func_openat=no ./configure

  2) more work: add a configure-time test for a working fstatat, and
    if it's not available, link with the replacement function that is
    currently included unconditionally via the definition in openat.c.

Since this is for a kernel that is old, but not *that* old, (i.e., the
problem affects only the few kernels, that had incomplete openat support)
spending time on #2 does not seem worthwhile to me.

Message #70 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: Adam Conrad <adconrad@0c3.net>, 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Tue, 26 Feb 2008 20:22:58 -0500

On Fri, Feb 15, 2008 at 05:09:31PM -0700, you wrote:
>So, this seems to be restricted to either older kernels, or kernels with
>a specific config.

Since my last email I've had my primary development machine die from bad 
capacitors, replaced it, put a new test qemu environment together, tried 
a variety of kernels working backward from 2.6.24, 2.6.18, 2.6.17, 
2.6.16, and finally had a failure with 2.6.8. (woo-hoo!)

I compiled & ran the test.c I attached earlier in the bug log, and it 
fails with the same "function not implmented error" & strace. In my 
mind, that suggests that this is a bug in the libc syscall emulation 
code rather than coreutils, and my inclination is to reassign to libc. 
Anyone have any other thoughts? (One other thought is that it should 
simply be downgraded since this shouldn't fail on any kernel released 
since etch, and we don't support skipping releases.)

(As for reassigning, the logic is that coreutils checks to see if 
unlinkat(2) is present in libc, and it is, so coreutils uses that rather 
than its own emulation code. That function works at build time, and 
works at run time if /proc is mounted. If proc is unmounted, unlinkat(2) 
suddenly fails. I don't think it's reasonable to expect coreutils to 
work around that. I do think that coreutils *would* work around that if 
unlinkat(2) wasn't present.)

Reproducing this is fairly easy if you install sid into a qemu image & 
then install the 2.6.8-4-686 kernel from sarge, as long as your machine 
doesn't have puffed up and knocked over capacitors. :-)

Mike Stone

Message #75 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Lucas Nussbaum <lucas@lucas-nussbaum.net>

To: Michael Stone <mstone@debian.org>, 464118@bugs.debian.org

Cc: Adam Conrad <adconrad@0c3.net>, control@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Tue, 4 Mar 2008 19:11:25 +0100

severity 464118 important
thanks

On 26/02/08 at 20:22 -0500, Michael Stone wrote:
> Reproducing this is fairly easy if you install sid into a qemu image &  
> then install the 2.6.8-4-686 kernel from sarge, as long as your machine  
> doesn't have puffed up and knocked over capacitors. :-)

So the etch kernel is not affected? If not, it's not RC, I think, since
we don't support sarge->lenny upgrades. In the meantime, I'm downgrading
the severity, which should allow coreutils to get into testing. We can
always increase it back if it's really considered RC.
-- 
| Lucas Nussbaum
| lucas@lucas-nussbaum.net   http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr             GPG: 1024D/023B3F4F |

Message #82 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@mathom.us>

To: Lucas Nussbaum <lucas@lucas-nussbaum.net>

Cc: 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Tue, 04 Mar 2008 13:34:39 -0500

On Tue, Mar 04, 2008 at 07:11:25PM +0100, you wrote:
>So the etch kernel is not affected? If not, it's not RC, I think, since
>we don't support sarge->lenny upgrades. In the meantime, I'm downgrading
>the severity, which should allow coreutils to get into testing. We can
>always increase it back if it's really considered RC.

I'm thinking about looking into what it would take to make coreutils 
call its internal fallback code (the code which would be used if 
removeat wasn't found at build time) if it gets a function not 
implemented error. I'm really on the fence with this one; part of me 
says that libc should do it, and part of me says that libc shouldn't use 
insecure emulation code in a function which was created largely for 
security reasons. But yeah, it's probably reasonable at this point to 
downgrade this since the buildds finally built the package.

Mike Stone

Message #87 received at 464118@bugs.debian.org (full text, mbox, reply):

From: James Westby <jw+debian@jameswestby.net>

To: 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Thu, 27 Mar 2008 11:37:33 +0000

Hi,

I was looking at

https://bugs.edge.launchpad.net/ubuntu/+source/coreutils/+bug/192239

and saw that the debugging was done in the Debian bug report.

You appear to have a handle on what the problem is now, and you
mention a few possible fixes, have you thought any more about
the issue?

Thanks,

James

Message #92 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: James Westby <jw+debian@jameswestby.net>, 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Thu, 27 Mar 2008 07:55:00 -0400

On Thu, Mar 27, 2008 at 11:37:33AM +0000, you wrote:
>https://bugs.edge.launchpad.net/ubuntu/+source/coreutils/+bug/192239
>
>and saw that the debugging was done in the Debian bug report.
>
>You appear to have a handle on what the problem is now, and you
>mention a few possible fixes, have you thought any more about
>the issue?

I'm mainly leaning toward ignoring it, since it isn't applicable to a 
supported configuration. (That is, it requires an unsupported kernel to 
trigger the problem, as well as not having /proc mounted, which is also 
non-standard.) One alternative is to look for a "function not supported" 
return code from the *at functions, and fall back to the functions which 
coreutils would have used if there were no *at functions in the first 
place. The patch to do that would be fairly invasive, though, because 
those functions are intended as replacements and aren't compiled if the 
libc function exists. (The patch would basically have to rename the 
functions and compile them unconditionally.) Another approach would be 
to get libc to use the same sort of fallback that the coreutils internal 
functions do, but I can understand why they might not want to do that. 
(The *at functions are intended to be more secure than the existing 
alternatives, and emulating them through insecure code would weaken the 
security assumptions.) If someone wants to propose a patch to implement
the above I would consider it.

Mike Stone

Message #97 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Lucas Nussbaum <lucas@lucas-nussbaum.net>

To: 464118@bugs.debian.org

Cc: control@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Fri, 4 Apr 2008 09:28:21 +0200

severity 464118 minor
retitle 464118 rm -r broken: Function not implemented (using coreutils 6.10 with a pre-etch kernel)
tags 464118 + wontfix
thanks

On 27/03/08 at 07:55 -0400, Michael Stone wrote:
> On Thu, Mar 27, 2008 at 11:37:33AM +0000, you wrote:
>> https://bugs.edge.launchpad.net/ubuntu/+source/coreutils/+bug/192239
>>
>> and saw that the debugging was done in the Debian bug report.
>>
>> You appear to have a handle on what the problem is now, and you
>> mention a few possible fixes, have you thought any more about
>> the issue?
>
> I'm mainly leaning toward ignoring it, since it isn't applicable to a  
> supported configuration. [...]

I agree. Marking it as such, and clarifying the title.
-- 
| Lucas Nussbaum
| lucas@lucas-nussbaum.net   http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr             GPG: 1024D/023B3F4F |

Message #108 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Martin Michlmayr <tbm@cyrius.com>

To: Lucas Nussbaum <lucas@lucas-nussbaum.net>

Cc: 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Wed, 3 Sep 2008 13:46:47 +0300

* Lucas Nussbaum <lucas@lucas-nussbaum.net> [2008-04-04 09:28]:
> severity 464118 minor
> retitle 464118 rm -r broken: Function not implemented (using coreutils 6.10 with a pre-etch kernel)
> tags 464118 + wontfix
> thanks

Actually, this seems to be wrong.  I just ran into this problem on an
Alpha box running etch when I was setting up an unstable chroot:

rm: cannot remove `/var/lib/dpkg/tmp.ci/control': Function not implemented

469:tbm@papa: ~] uname -a
Linux papa 2.6.18-5-alpha-generic #1 Fri Jun 1 00:16:02 UTC 2007 alpha GNU/Linux

So this should be RC.
-- 
Martin Michlmayr
http://www.cyrius.com/

Message #113 received at 464118@bugs.debian.org (full text, mbox, reply):

From: Martin Michlmayr <tbm@cyrius.com>

To: Lucas Nussbaum <lucas@lucas-nussbaum.net>

Cc: 464118@bugs.debian.org

Subject: Re: Bug#464118: rm -r broken: Function not implemented

Date: Wed, 3 Sep 2008 13:48:52 +0300

Ah, I see.  Mounting /proc indeeds fixes the problem.  So maybe it's
not RC, but at least important and should be documented.
-- 
Martin Michlmayr
http://www.cyrius.com/

Message #122 received at 464118-done@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: 464118-done@bugs.debian.org

Subject: Re: rm -r broken: Function not implemented (using coreutils 6.10 with a pre-etch kernel)

Date: Fri, 24 Oct 2014 09:04:33 -0400

The situation affected something obsolete enough that it's probably safe 
to just close the bug at this point--I don't think anyone is likely to 
be running into the problem again.

Mike Stone

Send a report that this bug log contains spam.