Debian Bug report logs - #707733
pygobject: FTBFS on kfreebsd

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Emilio Pozuelo Monfort <pochu@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: pygobject: FTBFS on kfreebsd

Date: Fri, 10 May 2013 20:41:41 +0200

Package: pygobject
Version: 3.8.1-2
Severity: serious

(CCing BSD porters, help wanted here)

pygobject currently fails to build on kfreebsd, see [1]

I've tried to debug this on falla. I can reproduce the hang somewhat reliably
by running:

dpkg-buildpackage

And if it doesn't hang or if you want to hang it again:

while true; do xvfb-run dh_auto_test --builddirectory=build-2.7; done

The hanging test is in test_overrides_gtk.py, but running with
TEST_FILES=test_overrides_gtk.py doesn't reproduce the hang so reliably here.

I've run gdb on the hanging python process and I get:

(gdb) thread apply all bt

Thread 1 (process 75189):
#0  0x000000080161ed4a in kevent () at ../sysdeps/unix/syscall-template.S:82
#1  0x0000000802a57bd7 in _kqueue_thread_func (arg=<optimized out>)
    at /build/buildd-glib2.0_2.36.1-2-kfreebsd-
amd64-CmfXXB/glib2.0-2.36.1/./gio/kqueue/kqueue-thread.c:226
#2  0x0000000800a91c4a in pthread_start_thread (arg=<optimized out>) at
manager.c:317
#3  0x0000000000000000 in ?? ()
(gdb)

Note that this is with libc0.1-dbg and libglib2.0-0-dbg installed.

After this I'm lost. Any help is welcome. Otherwise I'll just have to stop
running the whole test suite on kfreebsd, but I'd be sad to do that.

Thanks,
Emilio

[1] https://buildd.debian.org/status/fetch.php?pkg=pygobject&arch=kfreebsd-
amd64&ver=3.8.1-2&stamp=1368109683



-- System Information:
Debian Release: jessie/sid
  APT prefers experimental
  APT policy: (600, 'experimental'), (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.8-1-amd64 (SMP w/4 CPU cores)

Message #10 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Emilio Pozuelo Monfort <pochu@debian.org>

To: 707733@bugs.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Sat, 11 May 2013 17:08:45 +0200

On 10/05/13 20:41, Emilio Pozuelo Monfort wrote:
> Thread 1 (process 75189):
> #0  0x000000080161ed4a in kevent () at ../sysdeps/unix/syscall-template.S:82
> #1  0x0000000802a57bd7 in _kqueue_thread_func (arg=<optimized out>)
>      at /build/buildd-glib2.0_2.36.1-2-kfreebsd-
> amd64-CmfXXB/glib2.0-2.36.1/./gio/kqueue/kqueue-thread.c:226
> #2  0x0000000800a91c4a in pthread_start_thread (arg=<optimized out>) at
> manager.c:317
> #3  0x0000000000000000 in ?? ()
> (gdb)

Note that the glib2.0 test suite currently fails on kfreebsd. That may be 
unrelated, but it may be worth a look (plus it would be good to have it fixed too).

Message #15 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Emilio Pozuelo Monfort <pochu@debian.org>

To: 707733@bugs.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Sat, 11 May 2013 18:20:33 +0200

On 11/05/13 17:08, Emilio Pozuelo Monfort wrote:
> On 10/05/13 20:41, Emilio Pozuelo Monfort wrote:
>> Thread 1 (process 75189):
>> #0  0x000000080161ed4a in kevent () at ../sysdeps/unix/syscall-template.S:82
>> #1  0x0000000802a57bd7 in _kqueue_thread_func (arg=<optimized out>)
>>      at /build/buildd-glib2.0_2.36.1-2-kfreebsd-
>> amd64-CmfXXB/glib2.0-2.36.1/./gio/kqueue/kqueue-thread.c:226
>> #2  0x0000000800a91c4a in pthread_start_thread (arg=<optimized out>) at
>> manager.c:317
>> #3  0x0000000000000000 in ?? ()
>> (gdb)
>
> Note that the glib2.0 test suite currently fails on kfreebsd. That may be
> unrelated, but it may be worth a look (plus it would be good to have it fixed too).

I was just looking at the pygobject/armel build failure and came across this 
recent build log:

https://buildd.debian.org/status/fetch.php?pkg=pygobject&arch=armel&ver=3.7.92-2&stamp=1363793646

There the build hang in test_overrides_gtk.py too, so this may not be a kfreebsd 
specific issue (just triggered more easily there).

Emilio

Message #20 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Christoph Egger <christoph@debian.org>

To: Emilio Pozuelo Monfort <pochu@debian.org>

Cc: 707733@bugs.debian.org, debian-bsd@lists.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Sun, 12 May 2013 15:40:28 +0200

Emilio Pozuelo Monfort <pochu@debian.org> writes:
> Package: pygobject
> Version: 3.8.1-2
> Severity: serious
>
> (CCing BSD porters, help wanted here)
>
> pygobject currently fails to build on kfreebsd, see [1]
>
> I've tried to debug this on falla. I can reproduce the hang somewhat reliably
> by running:
>
> dpkg-buildpackage
>
> And if it doesn't hang or if you want to hang it again:
>
> while true; do xvfb-run dh_auto_test --builddirectory=build-2.7; done
>
> The hanging test is in test_overrides_gtk.py, but running with
> TEST_FILES=test_overrides_gtk.py doesn't reproduce the hang so reliably here.
>
> I've run gdb on the hanging python process and I get:
>
> (gdb) thread apply all bt
>
> Thread 1 (process 75189):
> #0  0x000000080161ed4a in kevent () at ../sysdeps/unix/syscall-template.S:82
> #1  0x0000000802a57bd7 in _kqueue_thread_func (arg=<optimized out>)
>     at /build/buildd-glib2.0_2.36.1-2-kfreebsd-
> amd64-CmfXXB/glib2.0-2.36.1/./gio/kqueue/kqueue-thread.c:226
> #2  0x0000000800a91c4a in pthread_start_thread (arg=<optimized out>) at
> manager.c:317
> #3  0x0000000000000000 in ?? ()
> (gdb)
>
> Note that this is with libc0.1-dbg and libglib2.0-0-dbg installed.
>
> After this I'm lost. Any help is welcome. Otherwise I'll just have to stop
> running the whole test suite on kfreebsd, but I'd be sad to do that.

Sounds like it could be similar to

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=706276

Looking at it right now

    Christoph

-- 
9FED 5C6C E206 B70A 5857  70CA 9655 22B9 D49A E731
Debian Developer | Lisp Hacker | CaCert Assurer

Message #25 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Emilio Pozuelo Monfort <pochu@debian.org>

To: Christoph Egger <christoph@debian.org>

Cc: 707733@bugs.debian.org, debian-bsd@lists.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Mon, 13 May 2013 21:05:28 +0200

On 12/05/13 15:40, Christoph Egger wrote:
> Emilio Pozuelo Monfort <pochu@debian.org> writes:
>> Package: pygobject
>> Version: 3.8.1-2
>> Severity: serious
>>
>> (CCing BSD porters, help wanted here)
>>
>> pygobject currently fails to build on kfreebsd, see [1]
>>
>> I've tried to debug this on falla. I can reproduce the hang somewhat reliably
>> by running:
>>
>> dpkg-buildpackage
>>
>> And if it doesn't hang or if you want to hang it again:
>>
>> while true; do xvfb-run dh_auto_test --builddirectory=build-2.7; done
>>
>> The hanging test is in test_overrides_gtk.py, but running with
>> TEST_FILES=test_overrides_gtk.py doesn't reproduce the hang so reliably here.
>>
>> I've run gdb on the hanging python process and I get:
>>
>> (gdb) thread apply all bt
>>
>> Thread 1 (process 75189):
>> #0  0x000000080161ed4a in kevent () at ../sysdeps/unix/syscall-template.S:82
>> #1  0x0000000802a57bd7 in _kqueue_thread_func (arg=<optimized out>)
>>      at /build/buildd-glib2.0_2.36.1-2-kfreebsd-
>> amd64-CmfXXB/glib2.0-2.36.1/./gio/kqueue/kqueue-thread.c:226
>> #2  0x0000000800a91c4a in pthread_start_thread (arg=<optimized out>) at
>> manager.c:317
>> #3  0x0000000000000000 in ?? ()
>> (gdb)
>>
>> Note that this is with libc0.1-dbg and libglib2.0-0-dbg installed.
>>
>> After this I'm lost. Any help is welcome. Otherwise I'll just have to stop
>> running the whole test suite on kfreebsd, but I'd be sad to do that.
>
> Sounds like it could be similar to
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=706276

That patch you have there seems to be work-arounding some deeper problem. The 
gdk_threads_enter/leave() calls are deprecated, and gdk/gtk calls must be done 
from the main thread, see:

https://developer.gnome.org/gdk3/stable/gdk3-Threads.html#gdk-threads-enter

If something like that fixes the hang here, then we'll at least have a clue, but 
it won't be the right fix.

>
> Looking at it right now
>
>      Christoph
>

It'd probably be a good idea looking at the glib2.0 test suite. Some tests there 
are failing on kfreebsd and they may be related to this. e.g. the spawn-async 
test (and if they are not it'd still be good to fix them. we plan on making the 
test suite fatal which will be good to find regressions and to make sure the 
ports are working fine)

BTW glib2.0 2.36.2 contains a fix for a hang in g_spawn_sync. It may be 
unrelated but I'll give it a try.

Emilio

Message #30 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Jeff Epler <jepler@unpythonic.net>

To: Emilio Pozuelo Monfort <pochu@debian.org>

Cc: Christoph Egger <christoph@debian.org>, 707733@bugs.debian.org, debian-bsd@lists.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Tue, 14 May 2013 08:13:17 -0500

[Message part 1 (text/plain, inline)]

Another bug that may be similar: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=671785

In that bug I remark that a problem with pthread_mutex_unlock can be
observed on linux with valgrind --tool=helgrind.  I haven't tried to
determine whether it's a similar problem here, but it might be worth
valgrinding it on Linux. (unfortunately I can't do this at the moment;
if I get a chance I'll report the results here)

Jeff

[signature.asc (application/pgp-signature, inline)]

Message #39 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Jeff Epler <jepler@unpythonic.net>

To: Christoph Egger <christoph@debian.org>

Cc: Emilio Pozuelo Monfort <pochu@debian.org>, 707733@bugs.debian.org, debian-bsd@lists.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Wed, 15 May 2013 21:25:06 -0500

[Message part 1 (text/plain, inline)]

valgrind (helgrind) on linux (sid amd64 chroot on wheezy amd64) didn't turn up
anything that looked too useful.  There were a number of diagnostics of
this general form:

==12158== Lock at 0x603E5C0 was first observed
==12158==    at 0x4C2EB32: pthread_mutex_init (in /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==12158==    by 0x6A644F6: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A64594: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A647C8: g_mutex_lock (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x67BDA97: g_type_create_instance (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x67A2757: ??? (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x67A4210: g_object_newv (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x67A485B: g_object_new (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x6568A48: ??? (in /usr/lib/libgirepository-1.0.so.1.0.0)
==12158==    by 0x6568F58: g_irepository_get_default (in /usr/lib/libgirepository-1.0.so.1.0.0)
==12158==    by 0x633BEEF: _wrap_g_irepository_get_default (pygi-repository.c:72)
==12158==    by 0x4B73E9: PyEval_EvalFrameEx (ceval.c:4005)
==12158== 
==12158== Possible data race during read of size 4 at 0xD2C3910 by thread #3
==12158== Locks held: none
==12158==    at 0x6A64BA9: g_private_set (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A48EF7: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x4C2E93D: ??? (in /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==12158==    by 0x4E3DE0D: start_thread (pthread_create.c:311)
==12158== 
==12158== This conflicts with a previous write of size 4 by thread #1
==12158== Locks held: 1, at address 0x603E5C0
==12158==    at 0x67BD9E0: g_type_create_instance (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x67A2757: ??? (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x67A4210: g_object_newv (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x67A485B: g_object_new (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0xC718EAA: ??? (in /usr/lib/x86_64-linux-gnu/libgtk-3.so.0.400.2)
==12158==    by 0x67BD93E: g_type_create_instance (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x67A2757: ??? (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x67A4210: g_object_newv (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158== 
==12158== Address 0xD2C3910 is 0 bytes inside a block of size 4 alloc'd
==12158==    at 0x4C2BEAB: malloc (in /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==12158==    by 0x6A6443D: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A64BA8: g_private_set (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A48EF7: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x4C2E93D: ??? (in /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==12158==    by 0x4E3DE0D: start_thread (pthread_create.c:311)

I don't know enough about the structure of glib to speculate as to whether this is expected or indicative of bad behavior.

and some of this general form:

==12158== Thread #1: pthread_cond_destroy: destruction of unknown cond var
==12158==    at 0x4C2D342: ??? (in /usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==12158==    by 0x6A64A0B: g_cond_clear (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x73670E8: ??? (in /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.3600.1)
==12158==    by 0x67A2577: g_object_unref (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0.3600.1)
==12158==    by 0x6A21DD7: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A22356: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A24F6F: g_main_context_dispatch (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A25267: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0x6A256D9: g_main_loop_run (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.3600.1)
==12158==    by 0xC8213B4: gtk_main (in /usr/lib/x86_64-linux-gnu/libgtk-3.so.0.400.2)
==12158==    by 0x7648E27: ffi_call_unix64 (in /usr/lib/x86_64-linux-gnu/libffi.so.6.0.1)
==12158==    by 0x764878F: ffi_call (in /usr/lib/x86_64-linux-gnu/libffi.so.6.0.1)

which may be a known bug in valgrind:  https://bugs.kde.org/show_bug.cgi?id=307082
If it's not a valgrind false positive, I do believe pthread_cond_destroy
on a cond variable already in an undefined state is undefined behavior
(but I couldn't find chapter and verse, just e.g., reports like this one
http://sourceware.org/bugzilla/show_bug.cgi?id=1473 where responders to
the bug say it is undefined behavior).  On the other hand, doing this in
a simple standalone program on kfreebsd-amd64 didn't provoke any
misbehavior out of the gate..

[signature.asc (application/pgp-signature, inline)]

Message #44 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Emilio Pozuelo Monfort <pochu@debian.org>

To: Jeff Epler <jepler@unpythonic.net>

Cc: Christoph Egger <christoph@debian.org>, 707733@bugs.debian.org, debian-bsd@lists.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Thu, 16 May 2013 11:33:37 +0200

Hurd seems to hang at the same place[1]. Perhaps that helps in determining where 
the bug may lie (e.g. if both Hurd and kfreebsd use the same pthread library 
implementation).

[1] 
https://buildd.debian.org/status/fetch.php?pkg=pygobject&arch=hurd-i386&ver=3.8.1-3&stamp=1368332988

Message #49 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Petr Salinger <Petr.Salinger@seznam.cz>

To: Emilio Pozuelo Monfort <pochu@debian.org>

Cc: Jeff Epler <jepler@unpythonic.net>, Christoph Egger <christoph@debian.org>, 707733@bugs.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Thu, 16 May 2013 15:53:03 +0200 (CEST)

> Hurd seems to hang at the same place[1]. Perhaps that helps in determining where
> the bug may lie (e.g. if both Hurd and kfreebsd use the same pthread library
> implementation).

They do not use the same pthread library implementation ...

Petr

Message #54 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Jeff Epler <jepler@unpythonic.net>

To: Emilio Pozuelo Monfort <pochu@debian.org>

Cc: Christoph Egger <christoph@debian.org>, 707733@bugs.debian.org, debian-bsd@lists.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Thu, 16 May 2013 22:37:55 -0500

[Message part 1 (text/plain, inline)]

OK, this seems crazy to me but I feel obliged to note it:

When I build 3.8.1-3 in /usr/src or /tmp/wat, I can observe the failure when I
subsequently 'make check' in build-2.7/tests.  When I build it in /tmp or
/tmp/wat/frugal-bonasfrarfsarfasrfasrf/pygobject-3.8.1 I do not.

However, I also note that I never saw a hang in 3.8.2-1 under a variety of
directory names.  When either version complets the testsuite, there is an
unexpected failure, though.

======================================================================
FAIL: test_main_loop (test_glib.TestGLib)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/wat/pygobject-3.8.1/tests/test_glib.py", line 95, in test_main_loop
    self.assertFalse(context.iteration(False))
AssertionError: True is not false

----------------------------------------------------------------------
Ran 877 tests in 10.171s

FAILED (failures=1, expected failures=4)

Jeff

[signature.asc (application/pgp-signature, inline)]

Message #59 received at 707733@bugs.debian.org (full text, mbox, reply):

From: Emilio Pozuelo Monfort <pochu@debian.org>

To: Jeff Epler <jepler@unpythonic.net>

Cc: Christoph Egger <christoph@debian.org>, 707733@bugs.debian.org, debian-bsd@lists.debian.org

Subject: Re: Bug#707733: pygobject: FTBFS on kfreebsd

Date: Fri, 17 May 2013 11:04:17 +0200

On 17/05/13 05:37, Jeff Epler wrote:
> OK, this seems crazy to me but I feel obliged to note it:
>
> When I build 3.8.1-3 in /usr/src or /tmp/wat, I can observe the failure when I
> subsequently 'make check' in build-2.7/tests.  When I build it in /tmp or
> /tmp/wat/frugal-bonasfrarfsarfasrfasrf/pygobject-3.8.1 I do not.

The thing I've noticed is that running `xvfb-run make check' hangs, but running 
TEST_FILES=test_overrides_gtk.py xvfb-run make check doesn't (I ran it in a loop 
for 83 times without hanging).

Send a report that this bug log contains spam.