Debian Bug report logs - #561225
coreutils: cp --reflink fails with Btrfs file system

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Jason White <jason@jasonjgw.net>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: coreutils: cp --reflink fails with Btrfs file system

Date: Tue, 15 Dec 2009 10:19:41 +0000

Package: coreutils
Version: 8.0-2
Severity: normal


jason@vrtl:~$ touch testfile
jason@vrtl:~$ cp --reflink testfile testfile2
cp: failed to clone `testfile2': Invalid argument

I am running kernel 2.6.32 with a Btrfs root file system under KVM, x86-64
architecture.

It was suggested on the Btrfs mailing list that I test file cloning by running
the bcp demonstration script provided as part of btrfs-progs (available from
a Btrfs Git repository). In my test, bcp was able to clone a file,
suggesting that this may be a Coreutils bug rather than a kernel issue.

-- System Information:
Debian Release: squeeze/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (800, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-trunk-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages coreutils depends on:
ii  libacl1                       2.2.49-1   Access control list shared library
ii  libattr1                      1:2.4.44-1 Extended attribute shared library
ii  libc6                         2.10.2-2   GNU C Library: Shared libraries
ii  libselinux1                   2.0.89-4   SELinux runtime shared libraries

coreutils recommends no packages.

coreutils suggests no packages.

-- no debconf information

Message #10 received at 561225@bugs.debian.org (full text, mbox, reply):

From: Jim Meyering <jim@meyering.net>

To: Jason White <jason@jasonjgw.net>

Cc: 561225@bugs.debian.org, bug-coreutils@gnu.org

Subject: Re: Bug#561225: coreutils: cp --reflink fails with Btrfs file system

Date: Tue, 15 Dec 2009 14:00:01 +0100

Jason White wrote:
> Package: coreutils
> Version: 8.0-2
>
> jason@vrtl:~$ touch testfile
> jason@vrtl:~$ cp --reflink testfile testfile2
> cp: failed to clone `testfile2': Invalid argument
>
> I am running kernel 2.6.32 with a Btrfs root file system under KVM, x86-64
> architecture.
>
> It was suggested on the Btrfs mailing list that I test file cloning by running
> the bcp demonstration script provided as part of btrfs-progs (available from
> a Btrfs Git repository). In my test, bcp was able to clone a file,
> suggesting that this may be a Coreutils bug rather than a kernel issue.

Thanks for the report.
Can you rerun that via strace and send the log at least
to the bug-coreutils list (Cc'd)?

    strace -o log cp --reflink testfile testfile2

Even better, can you do that using stock upstream coreutils-8.2,
assuming it still fails?

  ftp://ftp.gnu.org/gnu/coreutils/coreutils-8.2.tar.xz
  ftp://ftp.gnu.org/gnu/coreutils/coreutils-8.2.tar.gz

Message #15 received at 561225@bugs.debian.org (full text, mbox, reply):

From: Pádraig Brady <P@draigBrady.com>

To: Jim Meyering <jim@meyering.net>

Cc: Jason White <jason@jasonjgw.net>, bug-coreutils@gnu.org, 561225@bugs.debian.org

Subject: Re: Bug#561225: coreutils: cp --reflink fails with Btrfs file system

Date: Tue, 15 Dec 2009 13:25:33 +0000

On 15/12/09 13:00, Jim Meyering wrote:
> Jason White wrote:
>> Package: coreutils
>> Version: 8.0-2
>>
>> jason@vrtl:~$ touch testfile
>> jason@vrtl:~$ cp --reflink testfile testfile2
>> cp: failed to clone `testfile2': Invalid argument
>>
>> I am running kernel 2.6.32 with a Btrfs root file system under KVM, x86-64
>> architecture.
>>
>> It was suggested on the Btrfs mailing list that I test file cloning by running
>> the bcp demonstration script provided as part of btrfs-progs (available from
>> a Btrfs Git repository). In my test, bcp was able to clone a file,
>> suggesting that this may be a Coreutils bug rather than a kernel issue.

Note the bcp python script silently falls back to a standard copy,
so may be hiding the issue. Note you can get that behaviour with cp like:

cp --reflink=auto testfile testfile2

cheers,
Pádraig.

Message #20 received at 561225@bugs.debian.org (full text, mbox, reply):

From: Jason White <jason@jasonjgw.net>

To: Jim Meyering <jim@meyering.net>

Cc: 561225@bugs.debian.org, bug-coreutils@gnu.org

Subject: Re: Bug#561225: coreutils: cp --reflink fails with Btrfs file system

Date: Wed, 16 Dec 2009 10:21:34 +1100

[Message part 1 (text/plain, inline)]

Jim Meyering <jim@meyering.net> wrote:
> Can you rerun that via strace and send the log at least
> to the bug-coreutils list (Cc'd)?
> 
>     strace -o log cp --reflink testfile testfile2

It opens both files and then performs the ioctl() call:
ioctl(4, 0x40049409, 0x3)               = -1 EINVAL (Invalid argument)

> 
> Even better, can you do that using stock upstream coreutils-8.2,
> assuming it still fails?

This may not be necessary. If I run strace on bcp, it turns out that its
ioctl() call encounters the same error:

ioctl(3, 0x40049409, 0x4)               = -1 EINVAL (Invalid argument)
and thus the suggestion that bcp (by behaving like --reflink=auto) is masking
the issue appears to be exactly on point.

[cp.log (text/plain, attachment)]

Message #25 received at 561225@bugs.debian.org (full text, mbox, reply):

From: Jason White <jason@jasonjgw.net>

To: 561225@bugs.debian.org

Subject: Re: cp --reflink with Btrfs

Date: Wed, 16 Dec 2009 10:37:38 +1100

Josef Bacik  <josef@redhat.com> wrote:
>On Sun, Dec 13, 2009 at 12:29:03AM +0000, Jason White wrote:
>> I am testing a Btrfs root file system with Debian (kernel 2.6.32) under KVM.
>> 
>> jason@vrtl:~$ touch testfile
>> jason@vrtl:~$ cp --reflink testfile /tmp
>> cp: failed to clone `/tmp/testfile': Invalid argument
>> 
>> This is with GNU Coreutils 8.0 taken from debian Sid.
>> 
>> Is this a Coreutils issue, a Btrfs problem or something in my local
>> configuration?
>>
>
>Try using bcp, if that works then its likely a problem with coreutils.

After reporting this to Debian and engaging on follow-up discussion, it turns
out that bcp copies the data if the ioctl() call to clone the file fails, as
can be seen from the Python code (which I should have read, but didn't...).

Unfortunately the ioctl() call is failing both in bcp and in cp --reflink.

Here's partial strace output from the latter.

cp --reflink testfile testfile2

open("testfile", O_RDONLY)              = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
open("testfile2", O_WRONLY|O_CREAT|O_EXCL, 0644) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
ioctl(4, 0x40049409, 0x3)               = -1 EINVAL (Invalid argument)

Kernel 2.6.32 (debian Sid), x86-64 architecture.

Suggestions welcome.

Debian bug report:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561225

Message #30 received at 561225@bugs.debian.org (full text, mbox, reply):

From: Jim Meyering <jim@meyering.net>

To: Jason White <jason@jasonjgw.net>

Cc: 561225@bugs.debian.org, bug-coreutils@gnu.org

Subject: Re: Bug#561225: coreutils: cp --reflink fails with Btrfs file system

Date: Wed, 16 Dec 2009 09:46:25 +0100

Jason White wrote:
> Jim Meyering <jim@meyering.net> wrote:
>> Can you rerun that via strace and send the log at least
>> to the bug-coreutils list (Cc'd)?
>>
>>     strace -o log cp --reflink testfile testfile2
>
> It opens both files and then performs the ioctl() call:
> ioctl(4, 0x40049409, 0x3)               = -1 EINVAL (Invalid argument)
>
>> Even better, can you do that using stock upstream coreutils-8.2,
>> assuming it still fails?
>
> This may not be necessary. If I run strace on bcp, it turns out that its
> ioctl() call encounters the same error:
>
> ioctl(3, 0x40049409, 0x4)               = -1 EINVAL (Invalid argument)
> and thus the suggestion that bcp (by behaving like --reflink=auto) is masking
> the issue appears to be exactly on point.

Good to know it's not a problem with cp.
Thanks!

Message #35 received at 561225@bugs.debian.org (full text, mbox, reply):

From: Jason White <jason@jasonjgw.net>

To: Jim Meyering <jim@meyering.net>

Cc: 561225@bugs.debian.org, bug-coreutils@gnu.org

Subject: Re: Bug#561225: coreutils: cp --reflink fails with Btrfs file system

Date: Thu, 17 Dec 2009 10:00:58 +1100

From discussion on the Btrfs list, the reason for this behaviour is that, if
the file is empty (thus containing no data extents) the ioctl() call returns
EINVAL.

It has been suggested on the Btrfs list that it should return success instead,
so that the caller doesn't have to check for the empty file case.

The problem originally arose for me when I attempted cp -r --reflink directory
/tmp/destination
where directory contained what turned out, on inspection, to be empty files (I
didn't realize these were all empty files - it was a source code tree from a
project I downloaded.)

Message #40 received at 561225-close@bugs.debian.org (full text, mbox, reply):

From: Michael Stone <mstone@debian.org>

To: 561225-close@bugs.debian.org

Subject: Re: coreutils: cp --reflink fails with Btrfs file system

Date: Sat, 16 Jan 2010 14:58:20 -0500

It seems as though this isn't a coreutils bug, closing.

Mike Stone

Send a report that this bug log contains spam.