Debian Bug report logs - #751688
apt-utils: xz support is gone.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Christian Marillat <marillat@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: apt-utils: xz support is gone.

Date: Sun, 15 Jun 2014 18:10:39 +0200

Package: apt-utils
Version: 0.9.15.5
Severity: normal

Dear Maintainer,

Tested with 1.0.4 and reinstalled working 0.9.15.5

Packages::Compress seems to not work anymore with xz compression.

Only . (uncompressed), gzip and bzip2 seems to work, xz now return errors
(see at the bottom).

Same error with Sources::Compress

Christian


$ apt-ftparchive generate apt.conf
 dists/unstable/main/binary-amd64/:E: write, still have 32768 to write but
couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 32768 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)
E: write, still have 4096 to write but couldn't
E: IO to subprocess/file failed - write (0: Success)

E: Sub-process Compress child returned an error code (100)

-- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)

Kernel: Linux 3.15.0 (SMP w/8 CPU cores; PREEMPT)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages apt-utils depends on:
ii  libapt-inst1.5  0.9.15.5
ii  libapt-pkg4.12  0.9.15.5
ii  libc6           2.19-1
ii  libdb5.3        5.3.28-3
ii  libgcc1         1:4.9.0-6
ii  libstdc++6      4.9.0-6

apt-utils recommends no packages.

Versions of packages apt-utils suggests:
ii  xz-utils  5.1.1alpha+20120614-2

-- no debconf information

Message #14 received at 751688@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@ubuntu.com>

To: Christian Marillat <marillat@debian.org>, 751688@bugs.debian.org

Subject: Re: Bug#751688: apt-utils: xz support is gone.

Date: Fri, 4 Mar 2016 20:57:44 +0000

Control: tag -1 patch

On Sun, Jun 15, 2014 at 06:10:39PM +0200, Christian Marillat wrote:
> Packages::Compress seems to not work anymore with xz compression.
> 
> Only . (uncompressed), gzip and bzip2 seems to work, xz now return errors
> (see at the bottom).
> 
> Same error with Sources::Compress
> 
> Christian
> 
> 
> $ apt-ftparchive generate apt.conf
>  dists/unstable/main/binary-amd64/:E: write, still have 32768 to write but
> couldn't
> E: IO to subprocess/file failed - write (0: Success)
> E: write, still have 32768 to write but couldn't

The same bug hit the Ubuntu archive when we tried to add xz support
recently.  I finally got round to tracking this down.  The problem is
that the liblzma-based write code needs the same tweaks that the read
code already has to cope with the situation where lzma_code returns zero
the first time through because avail_out is zero, but will do more work
if called again.

The following patch ports the read tweaks to the write code as closely
as possible (including matching comments etc.), and it's working fine
for me now.

diff --git a/apt-pkg/contrib/fileutl.cc b/apt-pkg/contrib/fileutl.cc
index eae4d23..b7c7b23 100644
--- a/apt-pkg/contrib/fileutl.cc
+++ b/apt-pkg/contrib/fileutl.cc
@@ -1816,6 +1816,7 @@ public:
    }
    virtual ssize_t InternalWrite(void const * const From, unsigned long long const Size) override
    {
+      ssize_t Res;
       lzma->stream.next_in = (uint8_t *)From;
       lzma->stream.avail_in = Size;
       lzma->stream.next_out = lzma->buffer;
@@ -1826,9 +1827,21 @@ public:
       size_t const n = sizeof(lzma->buffer)/sizeof(lzma->buffer[0]) - lzma->stream.avail_out;
       size_t const m = (n == 0) ? 0 : fwrite(lzma->buffer, 1, n, lzma->file);
       if (m != n)
-	 return -1;
+      {
+	 Res = -1;
+	 errno = 0;
+      }
       else
-	 return Size - lzma->stream.avail_in;
+      {
+	 Res = Size - lzma->stream.avail_in;
+	 if (Res == 0)
+	 {
+	    // lzma run was okay, but produced no output…
+	    Res = -1;
+	    errno = EINTR;
+	 }
+      }
+      return Res;
    }
    virtual bool InternalWriteError() override
    {
@@ -2425,10 +2438,18 @@ bool FileFd::Write(const void *From,unsigned long long Size)
    while (Res > 0 && Size > 0)
    {
       Res = d->InternalWrite(From, Size);
-      if (Res < 0 && errno == EINTR)
-	 continue;
+
       if (Res < 0)
+      {
+	 if (errno == EINTR)
+	 {
+	    // trick the while-loop into running again
+	    Res = 1;
+	    errno = 0;
+	    continue;
+	 }
 	 return d->InternalWriteError();
+      }
 
       From = (char const *)From + Res;
       Size -= Res;

Thanks,

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

Message #21 received at 751688@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: Colin Watson <cjwatson@ubuntu.com>, 751688@bugs.debian.org

Subject: Re: Bug#751688: apt-utils: xz support is gone.

Date: Fri, 4 Mar 2016 23:28:47 +0100

On Fri, Mar 04, 2016 at 08:57:44PM +0000, Colin Watson wrote:
> Control: tag -1 patch
> 
> On Sun, Jun 15, 2014 at 06:10:39PM +0200, Christian Marillat wrote:
> > Packages::Compress seems to not work anymore with xz compression.
> > 
> > Only . (uncompressed), gzip and bzip2 seems to work, xz now return errors
> > (see at the bottom).
> > 
> > Same error with Sources::Compress
> > 
> > Christian
> > 
> > 
> > $ apt-ftparchive generate apt.conf
> >  dists/unstable/main/binary-amd64/:E: write, still have 32768 to write but
> > couldn't
> > E: IO to subprocess/file failed - write (0: Success)
> > E: write, still have 32768 to write but couldn't
> 
> The same bug hit the Ubuntu archive when we tried to add xz support
> recently.  I finally got round to tracking this down.  The problem is
> that the liblzma-based write code needs the same tweaks that the read
> code already has to cope with the situation where lzma_code returns zero
> the first time through because avail_out is zero, but will do more work
> if called again.
> 
> The following patch ports the read tweaks to the write code as closely
> as possible (including matching comments etc.), and it's working fine
> for me now.

Can you format it with format-patch, with commit message consisting of
a summary and a long description? That would be really nice.

-- 
Debian Developer - deb.li/jak | jak-linux.org - free software dev

When replying, only quote what is necessary, and write each reply
directly below the part(s) it pertains to (`inline'). Thank you.

Message #26 received at 751688@bugs.debian.org (full text, mbox, reply):

From: Colin Watson <cjwatson@ubuntu.com>

To: Julian Andres Klode <jak@debian.org>, 751688@bugs.debian.org

Subject: Re: Bug#751688: apt-utils: xz support is gone.

Date: Sat, 5 Mar 2016 01:20:29 +0000

[Message part 1 (text/plain, inline)]

On Fri, Mar 04, 2016 at 11:28:47PM +0100, Julian Andres Klode wrote:
> Can you format it with format-patch, with commit message consisting of
> a summary and a long description? That would be really nice.

Here you go.

-- 
Colin Watson                                       [cjwatson@ubuntu.com]

[0001-Fix-lzma-write-support-to-handle-try-again-case.patch (text/x-diff, attachment)]

Message #29 received at 751688-submitter@bugs.debian.org (full text, mbox, reply):

From: David Kalnischkies <david@kalnischkies.de>

To: 751688-submitter@bugs.debian.org

Subject: Bug#751688 in apt marked as pending

Date: Sun, 06 Mar 2016 12:10:48 +0000

Control: tag 751688 pending

Hello,

Bug #751688 in apt reported by you has been fixed in the Git repository. You can
see the commit message below, and you can check the diff of the fix at:

    https://anonscm.debian.org/cgit/apt/apt.git/diff/?id=9a63c3f

(this message was generated automatically based on the git commit message)
---
commit 9a63c3f480bcbc3232067237671b854d43a97236
Author: Colin Watson <cjwatson@ubuntu.com>
Date:   Fri Mar 4 22:17:12 2016 -0300

    Fix lzma write support to handle "try again" case
    
    The liblzma-based write code needs the same tweaks that the read code
    already has to cope with the situation where lzma_code returns zero the
    first time through because avail_out is zero, but will do more work if
    called again.
    
    This ports the read tweaks to the write code as closely as possible
    (including matching comments etc.).
    
    Closes: #751688

Message #36 received at 751688-close@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <jak@debian.org>

To: 751688-close@bugs.debian.org

Subject: Bug#751688: fixed in apt 1.2.5

Date: Sun, 06 Mar 2016 19:19:24 +0000

Source: apt
Source-Version: 1.2.5

We believe that the bug you reported is fixed in the latest version of
apt, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 751688@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Julian Andres Klode <jak@debian.org> (supplier of updated apt package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Sun, 06 Mar 2016 19:47:45 +0100
Source: apt
Binary: apt libapt-pkg5.0 libapt-inst2.0 apt-doc libapt-pkg-dev libapt-pkg-doc apt-utils apt-transport-https
Architecture: source
Version: 1.2.5
Distribution: unstable
Urgency: medium
Maintainer: APT Development Team <deity@lists.debian.org>
Changed-By: Julian Andres Klode <jak@debian.org>
Description:
 apt        - commandline package manager
 apt-doc    - documentation for APT
 apt-transport-https - https download transport for APT
 apt-utils  - package management related utility programs
 libapt-inst2.0 - deb package format runtime library
 libapt-pkg-dev - development files for APT's libapt-pkg and libapt-inst
 libapt-pkg-doc - documentation for APT development
 libapt-pkg5.0 - package management runtime library
Closes: 751688 812251 816691 816837
Changes:
 apt (1.2.5) unstable; urgency=medium
 .
   [ Daniel Kahn Gillmor ]
   * apt-key del should correctly handle keyids prefixed with 0x
 .
   [ David Kalnischkies ]
   * support APT::Get::Build-Dep-Automatic again in build-dep
   * add test for apt-key 0xKEY and use parameter expansion.
     Thanks to James McCoy for the suggestion. (Closes: 816691)
   * do not move not-failed pdiff-patches into CWD on failure (Closes: #816837)
   * get group again after potential remap in Source: parse.
     Thanks to Francesco Poli and Marc Haber for testdata. (Closes: 812251)
 .
   [ Colin Watson ]
   * Fix lzma write support to handle "try again" case (Closes: #751688)
 .
   [ Julian Andres Klode ]
   * Prevent double remapping of iterators and string views
   * debian/control: Set Standards-Version to 3.9.7
Checksums-Sha1:
 e1f0f2899572173dac6a1b905538654f3ec84380 2323 apt_1.2.5.dsc
 39bea5b453fb53eaf54e9e86ae6f1a838d4c2e2f 2021276 apt_1.2.5.tar.xz
Checksums-Sha256:
 555b532ea99b760fecc53a87b5060f2a0c356c9de5f0b9de42a806e0f622882d 2323 apt_1.2.5.dsc
 cc61eeeb32309d295f6348ba5e9c37bc54d5af7cdd1fa9e75abfbbd9c4c827b6 2021276 apt_1.2.5.tar.xz
Files:
 b68e6b93fa2f4f40f903c70ab4e2b608 2323 admin important apt_1.2.5.dsc
 19ee3d6c146c57e16a31f802540d1bf0 2021276 admin important apt_1.2.5.tar.xz

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJW3HvqAAoJENc8OeVlgLOGkPAP/2J8lMPhXtZGvqcd9GFEOzQt
h24zDcCkKu0tZ0XmMhSAvEMIL3V88fKfEA1Gf2TJm3rDwyBKio5FLAheP/+b+apa
xNPCQE6TuRvsyemyeGkN0eE6JPsGy72+RA+Ut3/hg104/grKPcW3ejc8uFuKxHl5
aBoNkDCKCoUe7pnP9iwJUzytj9od2gqzSjWJFoJr4OumhtSXZzqoGwuWckKp349J
wxnQrj4ODAhaX+vLn0n01BsFNwgb34q1v4TXtRDd3wtpX6s4ZwZM1VBLP2C6mYNw
3mMy8fkLvUcOqW1wlgvKDk4E5+t1KRzTGFfVkDIrkMK8J6vypOXz3LQ1KVplNB9Q
YrDViUDxeOzGor/1ZT9TD+kUDAI4ynTIWHV29UROiEOX9JJu1K5A/ZnnINN70OdF
H5yZSFA4lv6EYYvXYpdGkSHj2xVSyBBamAOtUV+H+E88u9mbT1TeLb1uqCNNoTdo
aJ966xzYuKKXPrT3h07EiQzh6jJ5/ih6OXgVE3w1tB0fA06hoNiwtnUcD1w3AU0Q
akHYIVgJn8eGlxbjswh2hBwL1OxxsBMMSiykZvOAGZacZfXbk7iBqb3C4WEztLRl
7RJ4HMhmy7A/Z8KlK2Xcfu+ytzc6QuF1lWFP2puBETV11LWqNLgmBySW/aRo5dVV
1efRhL785tiTAXpBKDNj
=jdyZ
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.