Debian Bug report logs - #505415
sv_chop() alignment problems on sparc

version graph

Package: perl; Maintainer for perl is Niko Tyni <ntyni@debian.org>; Source for perl is src:perl.

Reported by: Martin Zobel-Helas <zobel@ftbfs.de>

Date: Fri, 10 Oct 2008 19:03:04 UTC

Severity: important

Tags: fixed-upstream, upstream

Found in version perl/5.10.0-16

Fixed in version 5.10.1~rc2-1

Done: Ryan Niebur <ryanryan52@gmail.com>

Bug is archived. No further changes may be made.

Forwarded to http://rt.perl.org/rt3/Public/Bug/Display.html?id=60574

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Ivan Kohler <ivan-debian@420.am>:
Bug#501825; Package libjson-xs-perl. (Fri, 10 Oct 2008 19:03:07 GMT) Full text and rfc822 format available.

Acknowledgement sent to Martin Zobel-Helas <zobel@ftbfs.de>:
New Bug report received and forwarded. Copy sent to Ivan Kohler <ivan-debian@420.am>. (Fri, 10 Oct 2008 19:03:07 GMT) Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Martin Zobel-Helas <zobel@ftbfs.de>
To: submit@bugs.debian.org
Subject: libjson-xs-perl_2.23-1(sparc/unstable): FTBFS on sparc, fails in tests
Date: Fri, 10 Oct 2008 21:00:21 +0200
Package: libjson-xs-perl
Version: 2.23-1
Severity: serious

There was an error while trying to autobuild your package:

> Automatic build of libjson-xs-perl_2.23-1 on schroeder by sbuild/sparc 99.99
> Build started at 20081010-1421

[...]

> ** Using build dependencies supplied by package:
> Build-Depends: debhelper (>= 7), perl (>= 5.8.8)

[...]

> t/19_incr...............dubious
> 	Test returned status 0 (wstat 10, 0xa)
> DIED. FAILED test 697
> 	Failed 1/697 tests, 99.86% okay
> t/99_binary.............ok
> Failed Test Stat Wstat Total Fail  List of Failed
> t/19_incr.t    0    10   697    2  697
> Failed 1/21 test scripts. 1/25547 subtests failed.
> Files=21, Tests=25547, 22 wallclock secs (15.92 cusr +  0.42 csys = 16.34 CPU)
> Failed 1/21 test programs. 1/25547 subtests failed.
> make[1]: *** [test_dynamic] Error 255
> make[1]: Leaving directory `/build/buildd/libjson-xs-perl-2.23'
> dh_auto_test: command returned error code 512
> make: *** [build-stamp] Error 1
> dpkg-buildpackage: failure: debian/rules build gave error exit status 2

A full build log can be found at:
http://buildd.debian.org/build.php?arch=sparc&pkg=libjson-xs-perl&ver=2.23-1





Tags added: upstream Request was from "Angel Abad (Ikusnet SLL)" <angel@grupoikusnet.com> to control@bugs.debian.org. (Mon, 13 Oct 2008 11:33:05 GMT) Full text and rfc822 format available.

Noted your statement that Bug has been forwarded to http://rt.cpan.org/Public/Bug/Display.html?id=40005. Request was from "Angel Abad (Ikusnet SLL)" <angel@grupoikusnet.com> to control@bugs.debian.org. (Mon, 13 Oct 2008 11:33:06 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>:
Bug#501825; Package libjson-xs-perl. (Wed, 15 Oct 2008 20:42:10 GMT) Full text and rfc822 format available.

Acknowledgement sent to Niko Tyni <ntyni@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>. (Wed, 15 Oct 2008 20:42:10 GMT) Full text and rfc822 format available.

Message #14 received at 501825@bugs.debian.org (full text, mbox):

From: Niko Tyni <ntyni@debian.org>
To: Martin Zobel-Helas <zobel@ftbfs.de>, 501825@bugs.debian.org
Cc: debian-perl@lists.debian.org, debian-sparc@lists.debian.org
Subject: Re: Bug#501825: libjson-xs-perl_2.23-1(sparc/unstable): FTBFS on sparc, fails in tests
Date: Wed, 15 Oct 2008 23:39:52 +0300
On Fri, Oct 10, 2008 at 09:00:21PM +0200, Martin Zobel-Helas wrote:
> Package: libjson-xs-perl
> Version: 2.23-1
> Severity: serious
> 
> There was an error while trying to autobuild your package:
> 
> > Automatic build of libjson-xs-perl_2.23-1 on schroeder by sbuild/sparc 99.99
> > Build started at 20081010-1421

> > t/19_incr...............dubious
> > 	Test returned status 0 (wstat 10, 0xa)
> > DIED. FAILED test 697
> > 	Failed 1/697 tests, 99.86% okay

This is reproducible on sperger.d.o.

It's an alignment problem that only shows up with an optimization level
of at least -O2.

Program received signal SIGBUS, Bus error.
0xf7ec1134 in decode_json (string=0x192280, json=0x39f18, offset_return=0xff85b96c) at XS.xs:1443
1443      SvGROW (string, SvCUR (string) + 1); // should basically be a NOP
(gdb) bt
#0  0xf7ec1134 in decode_json (string=0x192280, json=0x39f18, offset_return=0xff85b96c) at XS.xs:1443
#1  0xf7ec2444 in XS_JSON__XS_incr_parse (my_perl=0x22008, cv=0x126d58) at XS.xs:1828
#2  0xf7dfab48 in Perl_pp_entersub () from /usr/lib/libperl.so.5.10
#3  0xf7df8f9c in Perl_runops_standard () from /usr/lib/libperl.so.5.10
#4  0xf7df4298 in perl_run () from /usr/lib/libperl.so.5.10
#5  0x00010b88 in main ()

The instruction that causes the bus error is a double-word load (ldd):

0xf7ec1130 <decode_json+132>:   ld  [ %l1 ], %g1
0xf7ec1134 <decode_json+136>:   ldd  [ %g1 + 8 ], %g2
0xf7ec1138 <decode_json+140>:   inc  %g2
0xf7ec113c <decode_json+144>:   cmp  %g3, %g2

The preprocessor output for the above SvGROW() macro invocation is

  (((XPV*) (string)->sv_any)->xpv_len < (((XPV*) (string)->sv_any)->xpv_cur + 1) ? Perl_sv_grow(((PerlInterpreter *)pthread_getspecific((*Perl_Gthr_key_ptr(((void *)0))))), string,((XPV*) (string)->sv_any)->xpv_cur + 1) : ((string)->sv_u.svu_pv));

and the above assembly is the compiled form of the first comparison
at -O2.  The compiler is using a double-word instruction to load both
xpv_cur and xpv_len in one go.

Now, this apparently requires that ((XPV*) (string)->sv_any)->xpv_cur
is aligned on a double-word (64-bit) boundary, which fails here:

$1 = {sv_any = 0x25714, sv_refcnt = 1, sv_flags = 536888326, sv_u = {svu_iv = 1662248, svu_uv = 1662248, 
    svu_rv = 0x195d28, svu_pv = 0x195d28 "[42]", svu_array = 0x195d28, svu_hash = 0x195d28, 
    svu_gp = 0x195d28}}
(gdb) print &((XPV*) (string)->sv_any)->xpv_cur
$2 = (STRLEN *) 0x2571c
(gdb) print &((XPV*) (string)->sv_any)->xpv_len
$3 = (STRLEN *) 0x25720

(Delving somewhat further, I see the 64-bit alignment of string->sv_any
 is lost in libperl's sv_chop(), invoked from incr_skip() at XS.xs:1855
 or so.)

This suggests to me that gcc is being too aggressive on optimizing here.
I don't see how it can consider the double word instruction safe.

Cc'ing the sparc porters list in the hope that somebody gets interested
in this.

I suppose a workaround is to use -O1 on sparc.
-- 
Niko Tyni   ntyni@debian.org




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>:
Bug#501825; Package libjson-xs-perl. (Wed, 15 Oct 2008 21:24:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Julien Cristau <jcristau@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>. (Wed, 15 Oct 2008 21:24:02 GMT) Full text and rfc822 format available.

Message #19 received at 501825@bugs.debian.org (full text, mbox):

From: Julien Cristau <jcristau@debian.org>
To: Martin Zobel-Helas <zobel@ftbfs.de>, 501825@bugs.debian.org, debian-perl@lists.debian.org, debian-sparc@lists.debian.org
Subject: Re: Bug#501825: libjson-xs-perl_2.23-1(sparc/unstable): FTBFS on sparc, fails in tests
Date: Wed, 15 Oct 2008 23:20:39 +0200
On Wed, Oct 15, 2008 at 23:39:52 +0300, Niko Tyni wrote:

> This is reproducible on sperger.d.o.
> 
> It's an alignment problem that only shows up with an optimization level
> of at least -O2.
> 
> Program received signal SIGBUS, Bus error.
> 0xf7ec1134 in decode_json (string=0x192280, json=0x39f18, offset_return=0xff85b96c) at XS.xs:1443
> 1443      SvGROW (string, SvCUR (string) + 1); // should basically be a NOP
> (gdb) bt
> #0  0xf7ec1134 in decode_json (string=0x192280, json=0x39f18, offset_return=0xff85b96c) at XS.xs:1443
> #1  0xf7ec2444 in XS_JSON__XS_incr_parse (my_perl=0x22008, cv=0x126d58) at XS.xs:1828
> #2  0xf7dfab48 in Perl_pp_entersub () from /usr/lib/libperl.so.5.10
> #3  0xf7df8f9c in Perl_runops_standard () from /usr/lib/libperl.so.5.10
> #4  0xf7df4298 in perl_run () from /usr/lib/libperl.so.5.10
> #5  0x00010b88 in main ()
> 
> The instruction that causes the bus error is a double-word load (ldd):
> 
> 0xf7ec1130 <decode_json+132>:   ld  [ %l1 ], %g1
> 0xf7ec1134 <decode_json+136>:   ldd  [ %g1 + 8 ], %g2
> 0xf7ec1138 <decode_json+140>:   inc  %g2
> 0xf7ec113c <decode_json+144>:   cmp  %g3, %g2
> 
> The preprocessor output for the above SvGROW() macro invocation is
> 
>   (((XPV*) (string)->sv_any)->xpv_len < (((XPV*) (string)->sv_any)->xpv_cur + 1) ? Perl_sv_grow(((PerlInterpreter *)pthread_getspecific((*Perl_Gthr_key_ptr(((void *)0))))), string,((XPV*) (string)->sv_any)->xpv_cur + 1) : ((string)->sv_u.svu_pv));
> 
> and the above assembly is the compiled form of the first comparison
> at -O2.  The compiler is using a double-word instruction to load both
> xpv_cur and xpv_len in one go.
> 
> Now, this apparently requires that ((XPV*) (string)->sv_any)->xpv_cur
> is aligned on a double-word (64-bit) boundary, which fails here:
> 
[...]
> 
> This suggests to me that gcc is being too aggressive on optimizing here.
> I don't see how it can consider the double word instruction safe.
> 
__alignof__(XPV) is 8, so gcc is allowed to assume that any XPV is
64-bit aligned, as far as I can tell.  xpv_cur's offset is 8, so it
should also be 64-bit aligned.

Cheers,
Julien




Message #20 received at 501825-forwarded@bugs.debian.org (full text, mbox):

From: Niko Tyni <ntyni@debian.org>
To: Marc Lehmann <schmorp@schmorp.de>
Cc: 501825-forwarded@bugs.debian.org
Subject: JSON-XS 2.23 alignment issue on Debian/sparc
Date: Mon, 20 Oct 2008 22:52:30 +0300
Hi Marc,

as seen in http://bugs.debian.org/501825 the JSON-XS-2.23 test suite
fails for us on Debian/sparc with a SIGBUS at the end of 19_incr.t.

This is an alignment problem where gcc -O2 is using double word
instructions on a struct that should be 64-bit aligned but isn't.

The sv_chop() function called from incr_skip() is making
self->incr_text->sv_any point to a non-64-bit aligned address, and the
SvGROW macro in decode_json() then casts it into an XPV pointer that
should be 64-bit aligned, as seen with __alignof__().

Please see the bug report for some details.

My (admittedly limited) understanding of the issue is that it looks
like a bug in the Perl core rather than JSON-XS, but I'd like to
have your opinion on this. 

(We have Perl 5.10.0, with a few selected patches from bleadperl
 applied.)

Thanks for your work on JSON-XS,
-- 
Niko Tyni           ntyni@debian.org
Debian Perl Group




Information forwarded to debian-bugs-dist@lists.debian.org, Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>:
Bug#501825; Package libjson-xs-perl. (Wed, 12 Nov 2008 07:42:17 GMT) Full text and rfc822 format available.

Acknowledgement sent to Niko Tyni <ntyni@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Perl Group <pkg-perl-maintainers@lists.alioth.debian.org>. (Wed, 12 Nov 2008 07:42:18 GMT) Full text and rfc822 format available.

Message #25 received at 501825@bugs.debian.org (full text, mbox):

From: Niko Tyni <ntyni@debian.org>
To: Martin Zobel-Helas <zobel@ftbfs.de>, 501825@bugs.debian.org
Cc: Marc Lehmann <schmorp@schmorp.de>
Subject: Re: Bug#501825: libjson-xs-perl_2.23-1(sparc/unstable): FTBFS on sparc, fails in tests
Date: Wed, 12 Nov 2008 09:39:40 +0200
clone 501825 -1
reassign -1 perl 5.10.0-16
retitle -1 sv_chop() alignment problems on sparc
severity -1 important
thanks

On Wed, Oct 15, 2008 at 11:39:52PM +0300, Niko Tyni wrote:
> On Fri, Oct 10, 2008 at 09:00:21PM +0200, Martin Zobel-Helas wrote:
> > Package: libjson-xs-perl
> > Version: 2.23-1
> > Severity: serious
> > 
> > There was an error while trying to autobuild your package:
> > 
> > > Automatic build of libjson-xs-perl_2.23-1 on schroeder by sbuild/sparc 99.99
> > > Build started at 20081010-1421
> 
> > > t/19_incr...............dubious
> > > 	Test returned status 0 (wstat 10, 0xa)
> > > DIED. FAILED test 697
> > > 	Failed 1/697 tests, 99.86% okay

Here's a status update on this; real life is currently leaving me little
hack time, so it's proceeding quite slowly. It doesn't apply to Lenny,
though, so it's not very urgent.

As I wrote earlier, this is an alignment problem on sparc where gcc -O2 is
using double word instructions on a struct that should be 64-bit aligned
but isn't. The alignment is lost in libperl's sv_chop() function. As far
as I understand, this happens because of the 'OOK hack', which upgrades
the SV into an SvPVIV, moving the value into a non-aligned location.

I think it's clear that this is a bug in the Perl core rather than JSON-XS
itself, so cloning and reassigning. The build failure can be worked around
by lowering the optimization level on sparc to -O1 or -O0, disabling the
use of double-word instructions.

I've reproduced the bug with a 'pristine' non-Debian Perl 5.10.0. It's
gone in bleadperl; bisecting shows it was fixed with change 32836,
which reimplements the OOK hack in a different way that doesn't upgrade
the SV anymore. It's still present in the current maint-5.10 branch,
as of change 34715.

I'm currently working on a minimal testcase for a proper Perl bug report.
Ideally, this would be a simple C program using libperl and cutting JSON-XS
from the loop.

I've discussed this a bit with Marc Lehmann (Cc'd), the JSON-XS
author. Unfortunately bugs.debian.org is silently discarding his messages
because their envelope-sender is root@..., matching FROM_DAEMON, so the
correspondence is not visible in the bug log.
-- 
Niko Tyni   ntyni@debian.org




Bug 501825 cloned as bug 505415. Request was from Niko Tyni <ntyni@debian.org> to control@bugs.debian.org. (Wed, 12 Nov 2008 07:42:19 GMT) Full text and rfc822 format available.

Bug reassigned from package `libjson-xs-perl' to `perl'. Request was from Niko Tyni <ntyni@debian.org> to control@bugs.debian.org. (Wed, 12 Nov 2008 07:42:21 GMT) Full text and rfc822 format available.

Changed Bug title to `sv_chop() alignment problems on sparc' from `libjson-xs-perl_2.23-1(sparc/unstable): FTBFS on sparc, fails in tests'. Request was from Niko Tyni <ntyni@debian.org> to control@bugs.debian.org. (Wed, 12 Nov 2008 07:42:23 GMT) Full text and rfc822 format available.

Severity set to `important' from `serious' Request was from Niko Tyni <ntyni@debian.org> to control@bugs.debian.org. (Wed, 12 Nov 2008 07:42:23 GMT) Full text and rfc822 format available.

Removed annotation that Bug had been forwarded to http://rt.cpan.org/Public/Bug/Display.html?id=40005. Request was from Niko Tyni <ntyni@debian.org> to control@bugs.debian.org. (Sat, 15 Nov 2008 11:06:16 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Brendan O'Dea <bod@debian.org>:
Bug#505415; Package perl. (Sun, 16 Nov 2008 12:30:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Niko Tyni <ntyni@debian.org>:
Extra info received and forwarded to list. Copy sent to Brendan O'Dea <bod@debian.org>. (Sun, 16 Nov 2008 12:30:02 GMT) Full text and rfc822 format available.

Message #40 received at 505415@bugs.debian.org (full text, mbox):

From: Niko Tyni <ntyni@debian.org>
To: Martin Zobel-Helas <zobel@ftbfs.de>, 505415@bugs.debian.org
Cc: Marc Lehmann <schmorp@schmorp.de>
Subject: Re: Bug#505415: sv_chop() alignment problems on sparc
Date: Sun, 16 Nov 2008 14:26:52 +0200
[Message part 1 (text/plain, inline)]
forwarded 505415 http://rt.perl.org/rt3/Public/Bug/Display.html?id=60574
thanks

On Wed, Nov 12, 2008 at 09:39:40AM +0200, Niko Tyni wrote:

> > On Fri, Oct 10, 2008 at 09:00:21PM +0200, Martin Zobel-Helas wrote:
> > > Package: libjson-xs-perl
> > > Version: 2.23-1
> > > Severity: serious
> > > 
> > > There was an error while trying to autobuild your package:
> > > 
> > > > Automatic build of libjson-xs-perl_2.23-1 on schroeder by sbuild/sparc 99.99
> > > > Build started at 20081010-1421
> > 
> > > > t/19_incr...............dubious
> > > > 	Test returned status 0 (wstat 10, 0xa)
> > > > DIED. FAILED test 697
> > > > 	Failed 1/697 tests, 99.86% okay

> I'm currently working on a minimal testcase for a proper Perl bug report.
> Ideally, this would be a simple C program using libperl and cutting JSON-XS
> from the loop.

This turned out to be easier than I thought; the attached program crashes
on Perl 5.10.0 but not bleadperl when compiled with -O2 -DUSE_CHOP,
and on both 5.10.0 and blead when compiled without it.

Now reported as [perl #60574]:

 http://rt.perl.org/rt3/Public/Bug/Display.html?id=60574

-- 
Niko Tyni   ntyni@debian.org
[crash.c (text/x-csrc, attachment)]

Noted your statement that Bug has been forwarded to http://rt.perl.org/rt3/Public/Bug/Display.html?id=60574. Request was from Niko Tyni <ntyni@debian.org> to control@bugs.debian.org. (Sun, 16 Nov 2008 12:30:04 GMT) Full text and rfc822 format available.

Added tag(s) fixed-upstream. Request was from Niko Tyni <ntyni@debian.org> to control@bugs.debian.org. (Fri, 24 Jul 2009 19:36:03 GMT) Full text and rfc822 format available.

Reply sent to Ryan Niebur <ryanryan52@gmail.com>:
You have taken responsibility. (Wed, 02 Sep 2009 02:51:07 GMT) Full text and rfc822 format available.

Notification sent to Martin Zobel-Helas <zobel@ftbfs.de>:
Bug acknowledged by developer. (Wed, 02 Sep 2009 02:51:08 GMT) Full text and rfc822 format available.

Message #49 received at 505415-done@bugs.debian.org (full text, mbox):

From: Ryan Niebur <ryanryan52@gmail.com>
To: 505415-done@bugs.debian.org
Date: Tue, 1 Sep 2009 19:47:16 -0700
[Message part 1 (text/plain, inline)]
Source-Version: 5.10.1~rc2-1

the patch that Niko said (in the upstream bug) fixes this is
apparently in 5.10.1~rc2-1, so closing bug report.

-- 
_________________________
Ryan Niebur
ryanryan52@gmail.com
[signature.asc (application/pgp-signature, inline)]

Added indication that 505415 affects libjson-xs-perl and libdevel-declare-perl Request was from Niko Tyni <ntyni@debian.org> to control@bugs.debian.org. (Wed, 02 Sep 2009 08:39:03 GMT) Full text and rfc822 format available.

Removed indication that 505415 affects libjson-xs-perl and libdevel-declare-perl Added indication that 505415 affects 543573 Request was from Damyan Ivanov <dmn@debian.org> to control@bugs.debian.org. (Wed, 02 Sep 2009 15:30:18 GMT) Full text and rfc822 format available.

Bug archived. Request was from Debbugs Internal Request <owner@bugs.debian.org> to internal_control@bugs.debian.org. (Wed, 28 Oct 2009 07:31:18 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Mon Apr 21 10:50:55 2014; Machine Name: beach.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.