Debian Bug report logs - #904699
W: parser_rfc822: Iek! Don't find end of value! when using sid or testing

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Nicolai Lissner <bugreport@gnuffy.net>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: W: parser_rfc822: Iek! Don't find end of value! when using sid or testing

Date: Thu, 26 Jul 2018 20:30:09 +0000

Package: cdebootstrap
Version: 0.7.7+b1
Severity: important

Dear Maintainer,

when I try to use cdebootstrap with current sid or testing repository
it just fails with
W: parser_rfc822: Iek! Don't find end of value!

it still works fine with stable though.

Didn't do anything special, just 
# cdebootstrap sid sid/ 
and it downloaded Release and Packages.xz, it validated 
and then while parsing...
P: Parsing Packages
W: parser_rfc822: Iek! Don't find end of value!
E: Internal error: download

and the program stops here.

I guess this is actually a problem in the package list of sid and testing, but
as I found debootstrap is capable to run successfully I wonder what could be done
to make cdebootstrap run fine again, too, as I clearly prefer it.

-- System Information:
Debian Release: buster/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 4.16.1 (SMP w/8 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)
LSM: AppArmor: enabled

Versions of packages cdebootstrap depends on:
ii  debian-archive-keyring      2017.7
ii  gpgv                        2.2.8-3
ii  libbz2-1.0                  1.0.6-8.1
ii  libc6                       2.27-5
ii  libcurl3-gnutls             7.60.0-2
ii  libdebian-installer-extra4  0.116
ii  libdebian-installer4        0.116
ii  liblzma5                    5.2.2-1.3
ii  zlib1g                      1:1.2.11.dfsg-1

cdebootstrap recommends no packages.

Versions of packages cdebootstrap suggests:
ii  qemu-user-static  1:2.12+dfsg-3

-- no debconf information

Message #10 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Nicolai Lissner <bugreport@gnuffy.net>

To: 904699@bugs.debian.org

Date: Thu, 2 Aug 2018 16:41:35 +0200

Found the reason for this bug.

On July 8th a package called librust-winapi-dev was accepted in sid

It comes with a complete common-sense breaking "Provides:" line of
more than 57kB length, as it provides 1336 packages.
IMHO this is insane. 

However, cdebootstrap uses
libdebian-installer-0.116 to parse the package 
list and in libdebian-installer-0.116/src/parser_rfc822.c
I found:

#define READSIZE 16384

and I guess increasing this to 65536 will fix the problem
until someone else encouraged by what the debian rust team did
will use even longer lines :/

As the parser_rfc822.c is from the same author as cdebootstrap,
I won't open a separated bug on that.

Please fix as soon as possible. Thanks a bunch.

Message #15 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Nicolai Lissner <bugreport@gnuffy.net>

To: 904699@bugs.debian.org

Subject: patch with solution

Date: Sun, 5 Aug 2018 02:21:13 +0200

[Message part 1 (text/plain, inline)]

Meanwhile, I've tested this and it works fine for me.

patch attached.

[lib-di-increase_max_line_length.patch (text/x-patch, attachment)]

Message #20 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Cyril Brulebois <kibi@debian.org>

To: Nicolai Lissner <bugreport@gnuffy.net>

Cc: Debian Install System Team <debian-boot@lists.debian.org>, 904699@bugs.debian.org

Subject: Re: problem in libdebian-installer with fix

Date: Sun, 5 Aug 2018 13:13:09 +0200

[Message part 1 (text/plain, inline)]

Hi Nicolai,

Nicolai Lissner <bugreport@gnuffy.net> (2018-08-05):
> while examining the reasons for a bug with cdebootstrap I found
> the actual problem is in libdebian-installer (well it was fine before
> librust-winapi-dev entered sid) and sent a fix.
> 
> Please have a look at Bug #904699 for details.

FWIW you could have reassigned (or cloned + reassigned) the bug report
to the libdebian-installer package (best done by cc-ing us).

That's a very good catch, and indeed a rather sad situation. Given the
current size of that Provides line, I fear bumping from 16k to 64k might
only paper over the issue for a while, and that it might come back later
on.

(FWIW, the currently second longest line is 13786 “only”.)


Cheers,
-- 
Cyril Brulebois (kibi@debian.org)            <https://debamax.com/>
D-I release manager -- Release team member -- Freelance Consultant

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: 904699@bugs.debian.org

Cc: 901827@bugs.debian.org

Subject: Re: Your mail

Date: Tue, 07 Aug 2018 02:20:00 +0000

On Thu, 2 Aug 2018 16:41:35 +0200 Nicolai Lissner <bugreport@gnuffy.net> wrote:
> Found the reason for this bug.
> 
> On July 8th a package called librust-winapi-dev was accepted in sid
> 
> It comes with a complete common-sense breaking "Provides:" line of
> more than 57kB length, as it provides 1336 packages.
> IMHO this is insane. 
> 
> However, cdebootstrap uses
> libdebian-installer-0.116 to parse the package 
> list and in libdebian-installer-0.116/src/parser_rfc822.c
> I found:
> 
> #define READSIZE 16384
> 
> and I guess increasing this to 65536 will fix the problem
> until someone else encouraged by what the debian rust team did
> will use even longer lines :/
> 
> As the parser_rfc822.c is from the same author as cdebootstrap,
> I won't open a separated bug on that.
> 
> Please fix as soon as possible. Thanks a bunch.
> 

Sorry for the inconvenience. For context, the current Debian rust situation is a workaround for dpkg bug #901827.

In general though, arbitrary buffer sizes should be a thing of the past. Have you considered rewriting that code in Rust? :) Or perhaps less effort, just reusing whatever code dpkg itself uses to parse it.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

Message #30 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Marc Kleine-Budde <mkl@pengutronix.de>

To: 904699@bugs.debian.org

Subject: W: parser_rfc822: Iek! Don't find end of value!

Date: Fri, 7 Sep 2018 14:19:12 +0200

[Message part 1 (text/plain, inline)]

Hello,

using buster I just ran into the same problem. Will the READSIZE be
increased or can't we cdebootstrap until the rust problem is sorted out?

regards,
Marc

[signature.asc (application/pgp-signature, attachment)]

Message #37 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Diederik de Haas <didi.debian@cknow.org>

To: 554444@bugs.debian.org

Cc: Debian Install System Team <debian-boot@lists.debian.org>, 904699@bugs.debian.org

Subject: Ping?

Date: Thu, 02 May 2019 20:54:34 +0200

[Message part 1 (text/plain, inline)]

I just tried to install a Buster system with cdebootstrap(-static) and that 
fails because this bug is not fixed.
According to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=904699#15, 
increasing READSIZE from 16384 to 65536 'fixes' it.

I agree that 65536 is also just a random number, but so late in the freeze it 
may be warranted to do it, so cdebootstrap becomes usable for Buster.

This seems RC to me, but I let it up to the maintainers to actually do that.

Cheers,
  Diederik

[signature.asc (application/pgp-signature, inline)]

Message #42 received at 904699-close@bugs.debian.org (full text, mbox, reply):

From: Asbjørn Sloth Tønnesen <asbjorn@asbjorn.st>

To: 904699-close@bugs.debian.org

Subject: Fixed in libdebian-installer 0.119

Date: Tue, 7 May 2019 11:55:30 +0000

Hi,

Closing this bug, will create a wishlist bug in a moment
to remove the arbitrary limit.

Bastian has uploaded a fixed libdebian-installer,
and fixed #554444.

-- 
Best regards
Asbjørn Sloth Tønnesen.

Message #47 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Diederik de Haas <didi.debian@cknow.org>

To: 554444@bugs.debian.org

Cc: 904699@bugs.debian.org

Subject: Rebuild for cdebootstrap-static?

Date: Sat, 11 May 2019 13:30:04 +0200

[Message part 1 (text/plain, inline)]

Thanks for upping the limit in libdebian-installer :)

But doesn't it require a rebuild for cdebootstrap-static to pick up this 
change? If so, could you do or request it?

TIA,
  Diederik

[signature.asc (application/pgp-signature, inline)]

Message #52 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Diederik de Haas <didi.debian@cknow.org>

To: 929172@bugs.debian.org

Cc: 554444@bugs.debian.org, 904699@bugs.debian.org, "debian-release@lists.debian.org" <debian-release@lists.debian.org>, debian-boot@lists.debian.org

Subject: Same issue as already reported, and partially fixed

Date: Fri, 31 May 2019 21:16:08 +0200

[Message part 1 (text/plain, inline)]

Control: severity -1 serious

Hi,

This is the same issue as https://bugs.debian.org/904699, which is actually an 
issue in libdebian-installer (https://bugs.debian.org/554444).

The fix has actually been made. But the problem is that it needs an unblock ack 
from the d-i team (https://bugs.debian.org/928908), but after 2 weeks there is 
still no reply.
When that is done, (afaik) a rebuild of the package to pick up the change in 
libdebian-installer is needed (at least for cdebootstrap-static? which I'm 
using).

Raised the issue to be RC as cdebootstrap(-static) is pretty much useless as 
it is now if you want to install buster (or sid).
My previous attempts to push this towards a solution have been unsuccessful so 
far and I don't know what else I could do...

[signature.asc (application/pgp-signature, inline)]

Message #61 received at 904699@bugs.debian.org (full text, mbox, reply):

From: Paul Gevers <elbrus@debian.org>

To: Diederik de Haas <didi.debian@cknow.org>, 929172@bugs.debian.org

Cc: 554444@bugs.debian.org, 904699@bugs.debian.org, "debian-release@lists.debian.org" <debian-release@lists.debian.org>, debian-boot@lists.debian.org, Debian bugs control server <control@bugs.debian.org>

Subject: Re: Same issue as already reported, and partially fixed

Date: Fri, 31 May 2019 21:52:38 +0200

[Message part 1 (text/plain, inline)]

severity 554444 grave
merge 929172 904699
thanks

Hi Diederik,

On 31-05-2019 21:16, Diederik de Haas wrote:
> Control: severity -1 serious

Please don't use this if you CC multiple bugs that aren't all of the
same severity.

> The fix has actually been made. But the problem is that it needs an unblock ack 
> from the d-i team (https://bugs.debian.org/928908), but after 2 weeks there is 
> still no reply.

Please have patience. D-I is busy.

> When that is done, (afaik) a rebuild of the package to pick up the change in 
> libdebian-installer is needed (at least for cdebootstrap-static? which I'm 
> using).

Please file a binNMU bug already than (I didn't check if that is the
correct course of action, but better discuss that there).

Paul

[signature.asc (application/pgp-signature, attachment)]

Send a report that this bug log contains spam.