Debian Bug report logs - #631295
gvfs-backends: gvfsd-gphoto2 handles large image/video files very badly

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Jack Whitham <jack@cs.york.ac.uk>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: gvfs-backends: gvfsd-gphoto2 handles large image/video files very badly

Date: Wed, 22 Jun 2011 18:37:14 +0100

Package: gvfs-backends
Version: 1.6.4-3
Severity: important
Tags: upstream


This bug can be reproduced by 
1. recording a large video file (1000Mb+) using a digital camera,
2. attaching that camera to the PC,
3. open the camera folder that appears on the desktop, then 
find the video file using the File Browser (nautilus),
4. copy the file onto the desktop by drag and drop.


The symptoms are as follows:
1. The entire file is copied into memory before being written to
$HOME/Desktop, potentially causing a swap storm or even crashing the PC.
Using "top" you can observe the size of the gvfsd-gphoto2 process
increasing during the copy operation.
2. The file is apparently downloaded from the camera more than once.
3. The copy progress bar appears, but sits at 0% for some time.


I have investigated the first symptom and traced it to code in
gvfsbackendgphoto2.c within the gvfs-1.6.4 source package. 
The function "do_open_for_read_real" is responsible for copying each
file from the camera, which is done using code in libgphoto2.

The "gp_file_new" function is used to allocate a CameraFile object
to temporarily store the data. Unfortunately, "gp_file_new" stores the 
data entirely in memory: fine for images of a few megabytes, but entirely
unsuitable for gigabyte-sized video files. 

An examination of the command line tool "gphoto2", which also uses 
libgphoto2, suggests that "gp_file_new_from_fd" would be a more 
appropriate choice. When "gp_file_new_from_fd" is used instead,
the memory usage of gvfsd-gphoto2 remains reasonable.

However, this fix just reveals the second problem, namely that each 
file will be downloaded more than once -- before the progress
bar even begins to move. Again, this is not noticeable for small 
files, but it is a major annoyance for large copies. Furthermore,
large copies may never complete, as the operation appears to time out.

If the problem were limited to the first symptom, namely the apparent 
misuse of "gp_file_new", then I could easily suggest a patch. But it 
appears that there is a deeper design problem, namely that:
1. A file is apparently downloaded from the camera more than once.
2. The copy progress bar appears, but sits at 0% for some time, whilst
those copies complete.


Further information:
1. The latest upstream version of gvfsbackendgphoto2.c also uses 
gp_file_new. 
2. To my knowledge the bug is not open within Debian, but it has 
apparently been reported in Ubuntu (note same symptoms):
https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/782933
https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/627908




-- System Information:
Debian Release: 6.0.1
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/1 CPU core)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages gvfs-backends depends on:
ii  gvfs              1.6.4-3                userspace virtual filesystem - ser
ii  libarchive1       2.8.4-1                Single library to read/write tar, 
ii  libavahi-client3  0.6.27-2+squeeze1      Avahi client library
ii  libavahi-common3  0.6.27-2+squeeze1      Avahi common library
ii  libavahi-glib1    0.6.27-2+squeeze1      Avahi glib integration library
ii  libbluetooth3     4.66-3                 Library to use the BlueZ Linux Blu
ii  libc6             2.11.2-10              Embedded GNU C Library: Shared lib
ii  libcdio-cdda0     0.81-4                 library to read and control digita
ii  libcdio-paranoia0 0.81-4                 library to read digital audio CDs 
ii  libcdio10         0.81-4                 library to read and control CD-ROM
ii  libdbus-1-3       1.2.24-4               simple interprocess messaging syst
ii  libdbus-glib-1-2  0.88-2.1               simple interprocess messaging syst
ii  libexpat1         2.0.1-7                XML parsing C library - runtime li
ii  libgconf2-4       2.28.1-6               GNOME configuration database syste
ii  libglib2.0-0      2.24.2-1               The GLib library of C routines
ii  libgphoto2-2      2.4.6-3                gphoto2 digital camera library
ii  libgphoto2-port0  2.4.6-3                gphoto2 digital camera port librar
ii  libgudev-1.0-0    164-3                  GObject-based wrapper library for 
ii  libimobiledevice1 1.0.2-1                Library for communicating with the
ii  libplist1         1.3-2                  Library for handling Apple binary 
ii  libsmbclient      2:3.5.6~dfsg-3squeeze2 shared library for communication w
ii  libsoup-gnome2.4- 2.30.2-1               an HTTP library implementation in 
ii  libsoup2.4-1      2.30.2-1               an HTTP library implementation in 
ii  libxml2           2.7.8.dfsg-2+squeeze1  GNOME XML library

Versions of packages gvfs-backends recommends:
ii  gnome-keyring                 2.30.3-5   GNOME keyring services (daemon and

Versions of packages gvfs-backends suggests:
ii  obex-data-server              0.4.5-1+b1 D-Bus service for OBEX client and 

-- no debconf information

Message #10 received at 631295@bugs.debian.org (full text, mbox, reply):

From: Rolf Leggewie <debian-bugs@rolf.leggewie.biz>

To: 631295@bugs.debian.org

Subject: This is RC-quality

Date: Tue, 18 Sep 2012 04:27:46 +0900

severity 631295 critical
thanks

I'm setting this to critical as given sufficiently large files the bug
has the potential to crash other processes on the system or at least
severely slow them down.  One can easily see that a 4GB file which is
fairly common these days would crash either the gvfs daemon or other
processes or both on one of the common netbooks.  This problem seems to
be a flawed design decision made by upstream and it may not be possible
to get a fix in time for wheezy.  Nonetheless, I think this is RC.  The
problem doesn't only affect the gphoto2 backend but at least the DAV
backend as well.

Message #17 received at 631295@bugs.debian.org (full text, mbox, reply):

From: Michael Biebl <biebl@debian.org>

To: Rolf Leggewie <debian-bugs@rolf.leggewie.biz>, 631295@bugs.debian.org

Cc: control@bugs.debian.org

Subject: Re: Bug#631295: This is RC-quality

Date: Tue, 18 Sep 2012 00:08:20 +0200

[Message part 1 (text/plain, inline)]

severity 631295 important
thanks

On 17.09.2012 21:27, Rolf Leggewie wrote:
> severity 631295 critical
> thanks
> 
> I'm setting this to critical as given sufficiently large files the bug
> has the potential to crash other processes on the system or at least
> severely slow them down.  One can easily see that a 4GB file which is
> fairly common these days would crash either the gvfs daemon or other
> processes or both on one of the common netbooks.  This problem seems to
> be a flawed design decision made by upstream and it may not be possible
> to get a fix in time for wheezy.  Nonetheless, I think this is RC.  The

I don't think so. Downgrading to the original severity.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?

[signature.asc (application/pgp-signature, attachment)]

Message #24 received at 631295@bugs.debian.org (full text, mbox, reply):

From: Rolf Leggewie <debian-bugs@rolf.leggewie.biz>

To: Michael Biebl <biebl@debian.org>

Cc: 631295@bugs.debian.org

Subject: Re: Bug#631295: This is RC-quality

Date: Tue, 18 Sep 2012 23:57:13 +0900

On 18.09.2012 07:08, Michael Biebl wrote:
> On 17.09.2012 21:27, Rolf Leggewie wrote:
>> Nonetheless, I think this is RC.
>
> I don't think so.

OK

Would you please be so kind to explain how a voracious process consuming
all memory and forcing other applications on the machine to grind to a
halt or even crash does not fit "critical: makes unrelated software on
the system (or the whole system) break, or causes serious data loss"?

Message #29 received at 631295@bugs.debian.org (full text, mbox, reply):

From: althaser <althaser@gmail.com>

To: jack@cs.york.ac.uk

Cc: 631295@bugs.debian.org, debian-bugs@rolf.leggewie.biz, biebl@debian.org

Subject: RE: gvfs-backends: gvfsd-gphoto2 handles large image/video files very badly

Date: Mon, 10 Mar 2014 23:53:36 +0000

[Message part 1 (text/plain, inline)]

Hey,

Could you please still reproduce this issue with newer versions ?

thanks
regards
althaser

[Message part 2 (text/html, inline)]

Message #36 received at 631295@bugs.debian.org (full text, mbox, reply):

From: Rupert Millard <rupert@rupertmillard.net>

To: 631295@bugs.debian.org

Subject: gvfs-backends: gvfsd-gphoto2 handles large image/video files very badly

Date: Sun, 28 Dec 2014 13:17:54 +0000

I can confirm that this bug is still present in the gvfs-backends 
package in Wheezy, version 1.12.3-4

It has been fixed upstream:
https://bugzilla.gnome.org/show_bug.cgi?id=642814
https://git.gnome.org/browse/gvfs/commit/?id=86162bbe4b09f517b551ff1c9207a119e91ab733

I'm sorry, but I don't really understand how to apply the fix to the 
package myself - I have been reading about it all morning.

Thanks,

Rupert

Message #51 received at 631295-submitter@bugs.debian.org (full text, mbox, reply):

From: Laurent Bigonville <bigon@debian.org>

To: control@bugs.debian.org

Cc: 631295-submitter@bugs.debian.org

Subject: bug 631295 is forwarded to https://bugzilla.gnome.org/show_bug.cgi?id=642814, closing 631295 ...

Date: Sun, 28 Dec 2014 15:28:54 +0100

forwarded 631295 https://bugzilla.gnome.org/show_bug.cgi?id=642814
close 631295 1.22.2-1
close 631295 1.23.2-1
thanks

Hi,

This seems to be fixed in unstable/jessie.

Cheers,

Laurent Bigonville

Send a report that this bug log contains spam.