Debian Bug report logs - #523323
missing source code

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Bas Zoetekouw <bas@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: missing source code

Date: Thu, 09 Apr 2009 13:28:56 +0200

Package: openarena
Version: 0.8.1-2
Severity: serious

Some parts of the openarena source code seems to be missing from the
debian packages, in particular the files that are supposed to be in
code/cgame/.  These files contain the actual game logic, so I think
the compiled versions of these files end up in the openarena-data
package somewhere.
Still, it's the core part of the game, and I definately think it must
be included in the debian package.  Actually, it might even be
required by policy to build these files from use for the
openarena-data package.

Thanks,
Bas.

-- System Information:
Debian Release: 5.0
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.26-1-amd64 (SMP w/1 CPU core)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages openarena depends on:
ii  libc6                     2.7-18         GNU C Library: Shared libraries
ii  libcurl3-gnutls           7.18.2-8lenny2 Multi-protocol file transfer libra
ii  libgl1-mesa-glx [libgl1]  7.0.3-7        A free implementation of the OpenG
ii  libogg0                   1.1.3-4        Ogg Bitstream Library
pn  libopenal0a               <none>         (no description available)
ii  libsdl1.2debian           1.2.13-2       Simple DirectMedia Layer
ii  libvorbis0a               1.2.0.dfsg-3.1 The Vorbis General Audio Compressi
ii  libvorbisfile3            1.2.0.dfsg-3.1 The Vorbis General Audio Compressi
ii  openarena-data            0.8-0bas1      OpenArena game data

openarena recommends no packages.

openarena suggests no packages.

Message #10 received at 523323-close@bugs.debian.org (full text, mbox, reply):

From: Jon Dowland <jon+bts@alcopop.org>

To: 523323-close@bugs.debian.org

Subject: orig.tar.gz identical to upstream

Date: Sun, 28 Jun 2009 16:12:09 +0100

[Message part 1 (text/plain, inline)]

Hello, I have just downloaded the upstream source from
<http://openarena.ws/svn/source/081/> and compared the
unpacked tar.bz2 to the Debian orig.tar.gz.  The only
difference is a .svnignore file is present in the Debian
archive.

Therefore there is no missing source code.


-- 
Jon Dowland

[signature.asc (application/pgp-signature, inline)]

Message #15 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Bas Zoetekouw <bas@debian.org>

To: 523323@bugs.debian.org

Cc: control@bugs.debian.org

Subject: Re: Bug#523323 closed by Jon Dowland <jon+bts@alcopop.org> (orig.tar.gz identical to upstream)

Date: Sun, 28 Jun 2009 18:24:16 +0200

reopen 523323
thanks

You wrote:

> Hello, I have just downloaded the upstream source from
> <http://openarena.ws/svn/source/081/> and compared the
> unpacked tar.bz2 to the Debian orig.tar.gz.  The only
> difference is a .svnignore file is present in the Debian
> archive.
> Therefore there is no missing source code.

but as I reported before:

> Some parts of the openarena source code seems to be missing from the
> debian packages, in particular the files that are supposed to be in
> code/cgame/.  These files contain the actual game logic, so I think
> the compiled versions of these files end up in the openarena-data
> package somewhere.

So the code that is missing is 
http://openarena.ws/svn/source/code/cgame/

Ít's too bad that upstream isn't included these sources in their release
tarballs, but that's no excuse for Debian not to distribute the complete
source to the game, including the game logic.

Thanks,
Bas.

-- 
+--------------------------------------------------------------+
| Bas Zoetekouw      | Sweet day, so cool, so calm, so bright, |
|--------------------| The bridall of the earth and skie:      |
| bas@zoetekouw.net  | The dew shall weep thy fall tonight;    |
+--------------------|                    For thou must die.   |
                     +-----------------------------------------+

Message #22 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: 523323@bugs.debian.org

Subject: Re: Bug#523323: missing source code

Date: Sat, 1 Aug 2009 14:20:40 +0100

reassign 523323 openarena-data
thanks

If you believe that suitable source code for OpenArena game data is not
provided, that would be a bug in openarena-data (a separate source package).

Regards,
    S

Message #31 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: 523323@bugs.debian.org

Cc: bas@debian.org

Subject: Re: Bug#523323: missing source code

Date: Sat, 24 Oct 2009 15:28:00 +0100

[Message part 1 (text/plain, inline)]

On Sat, 01 Aug 2009 at 14:20:40 +0100, Simon McVittie wrote:
> If you believe that suitable source code for OpenArena game data is not
> provided, that would be a bug in openarena-data (a separate source package).

Unfortunately, it appears that this is correct:

* openarena-data contains .pk3 files (zip files)
* of those, pak0.pk3 contains .qvm files (the "cgame", "game" and "ui"
  modules), which are bytecode for a VM built in to the Quake 3 engine
* the .qvm files are compiled from C in http://www.openarena.ws/svn/source/
  using a non-free compiler (lcc)

Using gcc (main)
================

It should also possible to compile the cgame, game and ui modules as
native code, by copying them into the openarena source package and defining
BUILD_GAME_SO. I don't know how portable these modules are (the current
packaging dodges that issue by having them as platform-independent bytecode,
that was (presumably) compiled on i386), so this may cause build failures on
64-bit or big-endian architectures.

This would mean that openarena-data could be made suitable for main by
removing the sourceless QVM files - but see below for compatibility.

Using lcc (contrib)
===================

openarena-data could be made suitable for contrib by working out what the
corresponding source code is, providing it in the source package, and requiring
that an external q3lcc be used to build it; however, see "Compatibility"
below. This would also demote openarena (the binaries) to contrib.

(If doing this, it would be convenient if the required version of lcc was
uploaded to non-free.)

Compatibility
=============

Confusing things further, the Quake 3 network protocol checks for
compatibility by applying a simple checksum to the PK3 files. The checksum
is based on the CRC32s of the unzipped *contents* of the PK3 file, so just
unzipping and re-compressing the PK3 file shouldn't be a problem, but
recompiling the VMs would be, as far as I can tell - unless Debian has the
same "blessed" QVM files that are in upstream's svn, Debian openarena will be
incompatible.

Since we're building Openarena from source anyway, we control the source
code that computes the checksum, so it should be possible to have our
Openarena cheat, and claim to other platforms' Openarena builds that its QVMs
match the "blessed" upstream ones, even if they actually don't exist.

The best way to do this would probably be to include some sort of datafile
containing the desired fake checksums in openarena-data, and patch
openarena to look for them.

The checksums in question are:

* normal checksum: a checksum over the CRC32s of the zip file members
* pure-server checksum: a checksum over the "checksum feed" (an integer
  representing the ordered sequence of PK3s the server is using?) followed
  by the CRC32s of the zip file members

so the fake checksum data would have to be a list of CRC32s of zipfile members
to pretend to have. The checksumming expects them to be 4-byte little-endian,
so the file could just consist of the raw bytes of the CRCs, in LE order. I
have some untested code for this if you're interested in this approach.

Regards,
    Simon

[signature.asc (application/pgp-signature, inline)]

Message #36 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Bruno Kleinert <fuddl@debian.org>

To: Simon McVittie <smcv@debian.org>, 523323@bugs.debian.org

Cc: bas@debian.org

Subject: Re: Bug#523323: missing source code

Date: Thu, 29 Oct 2009 14:17:04 +0100

[Message part 1 (text/plain, inline)]

Am Samstag, den 24.10.2009, 15:28 +0100 schrieb Simon McVittie:
> On Sat, 01 Aug 2009 at 14:20:40 +0100, Simon McVittie wrote:
> > If you believe that suitable source code for OpenArena game data is not
> > provided, that would be a bug in openarena-data (a separate source package).
> 
> Unfortunately, it appears that this is correct:
> 
> * openarena-data contains .pk3 files (zip files)
> * of those, pak0.pk3 contains .qvm files (the "cgame", "game" and "ui"
>   modules), which are bytecode for a VM built in to the Quake 3 engine
> * the .qvm files are compiled from C in http://www.openarena.ws/svn/source/
>   using a non-free compiler (lcc)
Yes, 100% absolutely correct! :)


> Using gcc (main)
> ================
> 
> It should also possible to compile the cgame, game and ui modules as
> native code, by copying them into the openarena source package and defining
> BUILD_GAME_SO. I don't know how portable these modules are (the current
> packaging dodges that issue by having them as platform-independent bytecode,
> that was (presumably) compiled on i386), so this may cause build failures on
> 64-bit or big-endian architectures.
> 
> This would mean that openarena-data could be made suitable for main by
> removing the sourceless QVM files - but see below for compatibility.
> 
> Using lcc (contrib)
> ===================
> 
> openarena-data could be made suitable for contrib by working out what the
> corresponding source code is, providing it in the source package, and requiring
> that an external q3lcc be used to build it; however, see "Compatibility"
> below. This would also demote openarena (the binaries) to contrib.
> 
> (If doing this, it would be convenient if the required version of lcc was
> uploaded to non-free.)
I'm quite unhappy if we had to stuff OpenArena in contrib/non-free, so
I'd prefer trying to get the game logic compiled by gcc as shared
objects. ATM I'm wondering why upstream doesn't take that way and
instead ships precompiled stuff and keeps relying on a non-free build
tool..?!


> Compatibility
> =============
> 
> Confusing things further, the Quake 3 network protocol checks for
> compatibility by applying a simple checksum to the PK3 files. The checksum
> is based on the CRC32s of the unzipped *contents* of the PK3 file, so just
> unzipping and re-compressing the PK3 file shouldn't be a problem, but
> recompiling the VMs would be, as far as I can tell - unless Debian has the
> same "blessed" QVM files that are in upstream's svn, Debian openarena will be
> incompatible.
> 
> Since we're building Openarena from source anyway, we control the source
> code that computes the checksum, so it should be possible to have our
> Openarena cheat, and claim to other platforms' Openarena builds that its QVMs
> match the "blessed" upstream ones, even if they actually don't exist.
> 
> The best way to do this would probably be to include some sort of datafile
> containing the desired fake checksums in openarena-data, and patch
> openarena to look for them.
> 
> The checksums in question are:
> 
> * normal checksum: a checksum over the CRC32s of the zip file members
> * pure-server checksum: a checksum over the "checksum feed" (an integer
>   representing the ordered sequence of PK3s the server is using?) followed
>   by the CRC32s of the zip file members
> 
> so the fake checksum data would have to be a list of CRC32s of zipfile members
> to pretend to have. The checksumming expects them to be 4-byte little-endian,
> so the file could just consist of the raw bytes of the CRCs, in LE order. I
> have some untested code for this if you're interested in this approach.
I'd prefer this cheating way. Or maybe we can convince upstream to make
the engine check for network protocol versions only instead of doing the
whole checksum procedure. As you say, doing all this anti-cheat stuff in
free (!!) software is senseless anyway... or did I miss something?

Ok, for now I'll check out if I can make the engine compile the game
logic as shared objects in the hope we can get rid of any non-free build
tools.

Thank you very much for your helpful comments and work! - Fuddl

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Bruno Kleinert <fuddl@debian.org>

Cc: 523323@bugs.debian.org, bas@debian.org

Subject: Re: Bug#523323: missing source code

Date: Thu, 29 Oct 2009 14:09:31 +0000

On Thu, 29 Oct 2009 at 14:17:04 +0100, Bruno Kleinert wrote:
> Am Samstag, den 24.10.2009, 15:28 +0100 schrieb Simon McVittie:
> > Since we're building Openarena from source anyway, we control the source
> > code that computes the checksum, so it should be possible to have our
> > Openarena cheat, and claim to other platforms' Openarena builds that its QVMs
> > match the "blessed" upstream ones, even if they actually don't exist.
[...]
> > so the fake checksum data would have to be a list of CRC32s of zipfile members
> > to pretend to have. The checksumming expects them to be 4-byte little-endian,
> > so the file could just consist of the raw bytes of the CRCs, in LE order. I
> > have some untested code for this if you're interested in this approach.
> I'd prefer this cheating way.

See attached patch. Warning: it's totally untested, you (or I) should add some
printfs to assess whether it actually works.

My suggestion for the checksum blob would be to turn the list of desired
fake-CRCs into an array of integers in Perl syntax, and embed it in a
script that's run at openarena-data build time to produce the actual blob;
that'd make it not too difficult to update for new upstream versions.
Something like (a longer version of):

my @fake_crcs = (
    0x12345678,         # textures/foo.jpg
    0x87654321,         # vm/cgame.qvm (not in Debian, needs non-free compiler)
);

We could even have a script that's *not* run by the build, that takes a blessed
upstream pk3 file as input, scans through it with Archive::Zip, produces
that array of integers, and also produces a stripped version of the pk3 file
that lacks the QVMs. I can probably work on this sometime next week if nobody
gets there first.

As far as I can see, faking the checksums is probably only necessary for
pak0.pk3, but I could be wrong.

In the longer term, yes it'd be good to have upstream not do this, but I can
see that it's desirable to verify that everyone is using the same pk3s, as
a guard against invalid bug reports caused by mismatched versions (I doubt
that the engine copes well with a mismatch), so rather than stripping out
the checksumming, it'd probably be better to ask upstream to compile the
game code alongside the engine build and not ship QVMs at all.

I have a vague recollection that the game-code is not very portable (it
might well rely on the fact that, in practice, everyone always built it
using either lcc for a 32-bit VM, or a real C compiler for i386 Windows,
i386 Linux or PowerPC MacOS). It may need some fixes to be able to
build natively on platforms where int, long and pointer are different sizes,
for instance.

    Simon

Message #46 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Bruno Kleinert <fuddl@debian.org>, 523323@bugs.debian.org

Cc: bas@debian.org

Subject: Re: Bug#523323: missing source code

Date: Sun, 29 Nov 2009 17:43:42 +0000

[Message part 1 (text/plain, inline)]

On Thu, 29 Oct 2009 at 14:17:04 +0100, Bruno Kleinert wrote:
> Am Samstag, den 24.10.2009, 15:28 +0100 schrieb Simon McVittie:
> > Unfortunately, it appears that this is correct:
> > 
> > * openarena-data contains .pk3 files (zip files)
> > * of those, pak0.pk3 contains .qvm files (the "cgame", "game" and "ui"
> >   modules), which are bytecode for a VM built in to the Quake 3 engine
> > * the .qvm files are compiled from C in http://www.openarena.ws/svn/source/
> >   using a non-free compiler (lcc)
> Yes, 100% absolutely correct! :)

The source code for these *might* be in openarena-modSDK-0.8.1, as found in
http://www.openarena.ws/svn/source/081/openarena-modSDK-0.8.1.tar.bz2 (yes,
that's a tarball checked-in to Subversion...)

This tarball is not suitable for Debian as-is (it contains lcc, which is
non-free) but it's a start.

Running "make" in this tree produces (at least for me) files with these
SHA1sums:

smcv@reptile(sid)% ( cd build/release-linux-i386 && sha1sum -b */vm/*.qvm )
58b414ebf18e39a770f030164299c7fe822eb6bb *baseq3/vm/cgame.qvm
39f937662e927183d82c71d1063347d1bfab291d *baseq3/vm/qagame.qvm
f3d6968f008d5c9125379292e4fa68cdc23dcaf3 *baseq3/vm/ui.qvm
66392fcc7de452e38c8004b3c887ba070185becf *missionpack/vm/cgame.qvm
6e91088de0757ddb6d7e50b0c54e2ec95993641d *missionpack/vm/qagame.qvm
dce23cee6df160453d1a6ce31651539ebd35aef9 *missionpack/vm/ui.qvm

*/qagame.qvm are not byte-identical to the versions in OA svn, but the others
are:

smcv@reptile% sha1sum -b vm/* missionpack/vm/*        ~/src/debian/bugsquash/oa
58b414ebf18e39a770f030164299c7fe822eb6bb *vm/cgame.qvm
0e000ca4e1e5591c82b08494e2839589e1e2a98f *vm/qagame.qvm
f3d6968f008d5c9125379292e4fa68cdc23dcaf3 *vm/ui.qvm
66392fcc7de452e38c8004b3c887ba070185becf *missionpack/vm/cgame.qvm
24cb18f50875a5f22420bd2408011d833e48ff44 *missionpack/vm/qagame.qvm
dce23cee6df160453d1a6ce31651539ebd35aef9 *missionpack/vm/ui.qvm

I'll investigate this further.

If I can hack up the qagame sources to produce a QVM identical to upstream's,
or determine that the changes are insignificant, then we can consider this to
be suitable source code to compile native shared objects.

One caveat is that when I tried rsync'ing the "mod SDK" into the openarena
source package (which is upstream's "engine" tarball), I got differences in
some files that are common to the engine and the QVMs (the "mod SDK" and the
engine were both, independently, forked from ioQuake3). This is also something
that'll need further analysis.

So, I now have two directions in which to investigate:

Engine
======

* Does my patch work? (i.e. add debug printfs)
* Write a perl script as I described in my earlier mail, to strip the QVMs
  out of the PK3 and (as a side-effect) produce the file containing faked
  checksums

Game logic
==========

* How do we transform the mod SDK source into something from which the
  "canonical" upstream qagame.qvm can be compiled? If we can't, does it matter?
* Is it safe to use the mod SDK source in a subdirectory to build modules for 
  the engine (duplicating common files, to avoid having to reconcile the
  differences), or would it be better to reconcile them, or is it in fact
  *necessary* (for API/ABI compatibility) to reconcile them?
* Does it work with gcc on miscellaneous architectures? (I have a powerpc
  here, and since I run i386 with an amd64 kernel, I can hopefully make
  an amd64 chroot to test in)

In addition, someone needs to educate upstream about the correct use of svn
(tags, not checking in derived files, having a reproducible build process, and
similar considerations), and about the fact that what distributions really
care about is the source release, not the binary release. I'm not going to
do that, because I'd just end up flaming and alienating them...

Regards from the Cambridge BSP,
    Simon

[signature.asc (application/pgp-signature, inline)]

Message #51 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Bruno Kleinert <fuddl@debian.org>, 523323@bugs.debian.org

Cc: bas@debian.org

Subject: Re: Bug#523323: missing source code

Date: Sun, 29 Nov 2009 17:48:07 +0000

[Message part 1 (text/plain, inline)]

On Sun, 29 Nov 2009 at 17:43:42 +0000, Simon McVittie wrote:
> */qagame.qvm are not byte-identical to the versions in OA svn

Combining xxd and diff reveals that the only difference is the date, so we can
be fairly confident that this is, in fact, the source code.

    s

[signature.asc (application/pgp-signature, inline)]

Message #56 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Bruno Kleinert <fuddl@debian.org>, 523323@bugs.debian.org

Subject: Re: Bug#523323: missing source code

Date: Sun, 29 Nov 2009 19:35:36 +0000

[Message part 1 (text/plain, inline)]

On Sun, 29 Nov 2009 at 17:43:42 +0000, Simon McVittie wrote:
> One caveat is that when I tried rsync'ing the "mod SDK" into the openarena
> source package (which is upstream's "engine" tarball), I got differences in
> some files that are common to the engine and the QVMs (the "mod SDK" and the
> engine were both, independently, forked from ioQuake3). This is also something
> that'll need further analysis.

Where I say "only needed in the engine" below, it was verified by inserting a
#error at the top of the version included in the "mod SDK" :-)

Summary of differences between engine /code and "mod SDK" /code:

* code/botlib/*.c are only needed in the engine; some of *.h are needed, but
  they're not modified

* code/qcommon/*.[ch] aren't needed for the "mod SDK", apart from:
  - q_platform.h (not patched)
  - q_shared.h (will need reconciling)
    + mod SDK forces a standalone build; looks harmless to apply to engine
    + Q_isanumber and Q_isintegral added in the engine; looks harmless
  - q_shared.c (will need reconciling)
    + Q_isanumber and Q_isintegral added in the engine; looks harmless
  - q_math.c (not patched)

* code/client/*.[ch] is only needed in the engine, so it can be ignored,
  except:
  - keycodes.h (not patched)

* code/renderer/*.[ch] is only needed in the engine, except:
  - tr_types.h (not patched)

* code/ui/ui_shared.h needs patching
  + it just bumps NUM_CROSSHAIRS to 99, which looks harmless

So, pasting the mod SDK into the engine and building it natively seems
feasible.

    S

[signature.asc (application/pgp-signature, inline)]

Message #61 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Bruno Kleinert <fuddl@debian.org>, 523323@bugs.debian.org

Subject: Re: Bug#523323: missing source code

Date: Tue, 1 Dec 2009 02:54:46 +0000

[Message part 1 (text/plain, inline)]

I think I'm winning! I've deleted the QVMs from my pak0.pk3 and mp-pak0.pk3,
and I can play Openarena locally with and without "mission pack" mode by
using native shared objects for game logic, as long as I turn off sv_pure.

Changes made in <git+ssh://git.debian.org/git/users/smcv/openarena.git> (still
under development, please do not integrate yet):

* add a PKGLIBDIR make variable for an alternative load location for native
  shared objects, and set it to /usr/lib/games/openarena for Debian
* patch the game logic source code into the engine (merging a few files that
  are apparently used in both)
* build shared objects for the game logic "plugins" and install them to
  /usr/lib/games/openarena/baseoa and .../missionpack
* install the shared objects in openarena-server and make openarena depend on
  it, to avoid having to add a third architecture-any package (openarena-server
  is less than a megabyte, the game logic is 2.6M and the game data is over
  300M, so I don't think wasting space by having the dedicated server installed
  on client systems is a concern)
* sync up the Makefile with the lists of source files the "mod SDK" uses
  (the mod SDK integrates Unlagged, and builds some things differently,
  perhaps because its baseline version of ioQuake3 is different?)
* force the use of shared objects, even on pure servers (because soon we
  won't have QVMs)

Things still to do:

* Make sure this compiles and runs OK on powerpc and amd64, and at least
  compiles on some other architectures
* Work out why I get "impure" errors connecting to myself with sv_pure 1,
  and work around it somehow
* Apply and test the patch I wrote for faking pure-server checksums
* Write the script I suggested to strip out the QVMs from the PK3 while
  emitting a fake-checksums file as a side-effect
* Apply that script to get openarena-data 0.8.1+dfsg

    S

[signature.asc (application/pgp-signature, inline)]

Message #66 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: 523323@bugs.debian.org

Subject: Re: Bug#523323: openarena-data: missing source code

Date: Thu, 3 Dec 2009 02:30:39 +0000

[Message part 1 (text/plain, inline)]

tags 523323 + patch
block 523323 by 559240
thanks

So it took a while, and will still need longer while waiting for #559240 to
get to testing... but here's a patch for openarena-data to get rid of the
sourceless QVMs.

As well as applying the attached, it's also necessary to repack the
orig.tar.gz, replacing pak0.pk3 and mp-pak0.pk3 with versions that have are
output by the included strip-qvms.pl. This will truncate any vm/*.qvm to
0 bytes long.

The openarena patches supplied in #559240 are necessary to make the resulting
openarena-data package actually work. I've tested the following scenarios,
with sv_pure enabled in all cases:

* i386 server with unpatched packages; i386, powerpc and amd64 clients with
  patched packages
* amd64 server with patched packages; powerpc client with unpatched packages

Regards,
    Simon

[openarena-data-0.8.1+dfsg1-0.1-nmu.diff (text/x-diff, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #75 received at 523323@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: 523323@bugs.debian.org

Subject: Re: Bug#523323: openarena-data: missing source code

Date: Sun, 18 Jul 2010 19:41:11 +0100

On Thu, 03 Dec 2009 at 02:30:39 +0000, Simon McVittie wrote:
> As well as applying the attached, it's also necessary to repack the
> orig.tar.gz, replacing pak0.pk3 and mp-pak0.pk3 with versions that have are
> output by the included strip-qvms.pl. This will truncate any vm/*.qvm to
> 0 bytes long.

I've been reconsidering the approach I was using before, since it's pretty
ugly, and every time we change the format of the extra info provided, we'd
need to re-upload openarena-data (yuck). So, I think it's worth getting it
right first time!

The approach I'm now planning is:

* QVMs have a magic number in the header, with two possible values identifying
  variations on the format. Add a third value, which means "load native code
  instead", and patch the engine to understand it

* Write a C program that can construct QVM files with a desired CRC32 by brute
  force. Make some QVMs with the following contents:

  - the magic number to mean "go and load the native code"
  - whatever random rubbish the script generated to get the desired CRC32

  On my laptop, it takes seconds to brute-force this using zlib with the
  attached program:

  searching for xxxx such that NTVExxxx has CRC32 0x00031337
  collision found: xxxx = 0x6fb2d782 (in this machine's endianness, whatever that is)
  ./a.out 0x31337  26.11s user 0.00s system 99% cpu 26.139 total

* Replace each QVM with a brute-forced one constructed as above

* While we're repacking the source tarball anyway, it would be more transparent
  if we decompose the PK3 files into their component files plus an (ordered!)
  file list, used that as the orig tarball, and zip up the PK3 file during
  build; we could also check for mismatches during build

Regards,
    Simon

Message #80 received at 523323-close@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: 523323-close@bugs.debian.org

Subject: Bug#523323: fixed in openarena-data 0.8.1+dfsg1-1

Date: Thu, 22 Jul 2010 16:04:59 +0000

Source: openarena-data
Source-Version: 0.8.1+dfsg1-1

We believe that the bug you reported is fixed in the latest version of
openarena-data, which is due to be installed in the Debian FTP archive:

openarena-data_0.8.1+dfsg1-1.debian.tar.gz
  to main/o/openarena-data/openarena-data_0.8.1+dfsg1-1.debian.tar.gz
openarena-data_0.8.1+dfsg1-1.dsc
  to main/o/openarena-data/openarena-data_0.8.1+dfsg1-1.dsc
openarena-data_0.8.1+dfsg1-1_all.deb
  to main/o/openarena-data/openarena-data_0.8.1+dfsg1-1_all.deb
openarena-data_0.8.1+dfsg1.orig.tar.bz2
  to main/o/openarena-data/openarena-data_0.8.1+dfsg1.orig.tar.bz2



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 523323@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Simon McVittie <smcv@debian.org> (supplier of updated openarena-data package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Thu, 22 Jul 2010 10:47:38 +0100
Source: openarena-data
Binary: openarena-data
Architecture: source all
Version: 0.8.1+dfsg1-1
Distribution: unstable
Urgency: low
Maintainer: Debian Games Team <pkg-games-devel@lists.alioth.debian.org>
Changed-By: Simon McVittie <smcv@debian.org>
Description: 
 openarena-data - OpenArena game data
Closes: 523323
Changes: 
 openarena-data (0.8.1+dfsg1-1) unstable; urgency=low
 .
   [ Iain Lane ]
   * debian/control: Remove myself from Uploaders
 .
   [ Simon McVittie ]
   * Add myself to Uploaders
   * Set source format to 3.0 (quilt)
   * Repack upstream tarball, unzipping the PK3 files for greater transparency
     and removing QVM files that cannot be compiled with a Free compiler
     (closes: #523323)
   * At build time, repack the PK3 files
   * At build time, replace the missing QVM files with a magic number that
     causes openarena (>= 0.8.1-8) to load native-code equivalents, and has
     the same CRC-32 as the missing file for network compatibility
   * Breaks: openarena-data (<< 0.8.1-8~) due to those changes
   * Split README.Debian into README.Debian and README.source, with the latter
     going into more detail about the process
   * Add an empty debian/watch explaining why we're not uscan-compatible
Checksums-Sha1: 
 28e63da792bfb100a9f1cfb5eb766da51d79af6c 2063 openarena-data_0.8.1+dfsg1-1.dsc
 5a6979b65a45afbf3fed206fe299e3f2c91bd640 288069627 openarena-data_0.8.1+dfsg1.orig.tar.bz2
 9429147963cf72ade7eb879281dad6e29fb57b42 52441 openarena-data_0.8.1+dfsg1-1.debian.tar.gz
 0df5ecf84d82976786a9e86aedfa0babe762539f 315414174 openarena-data_0.8.1+dfsg1-1_all.deb
Checksums-Sha256: 
 5a9ca57e0295f40db599ec172f5e83f899169a69654199a6a1fffeed3a712861 2063 openarena-data_0.8.1+dfsg1-1.dsc
 e2e60e5420aced7d738d77bc102a288944a54bec5d1756a9549b44577529526e 288069627 openarena-data_0.8.1+dfsg1.orig.tar.bz2
 0b99ca3a21294abedf60c04b7b57b66402156308a042a3f5845b1c5569be19a3 52441 openarena-data_0.8.1+dfsg1-1.debian.tar.gz
 b8be96ca985327376698be8fdabdb6b1e9c234e5596d7f94f172d1490c036e8d 315414174 openarena-data_0.8.1+dfsg1-1_all.deb
Files: 
 8120feb6ca885520213b6fb8d9d6c450 2063 games optional openarena-data_0.8.1+dfsg1-1.dsc
 365897051bf168c667c09773da40047d 288069627 games optional openarena-data_0.8.1+dfsg1.orig.tar.bz2
 ae69495b6c0289177dc942693b452117 52441 games optional openarena-data_0.8.1+dfsg1-1.debian.tar.gz
 54679ef5307a3601f621d4f91644d325 315414174 games optional openarena-data_0.8.1+dfsg1-1_all.deb

-----BEGIN PGP SIGNATURE-----

iQIVAwUBTEgrBE3o/ypjx8yQAQjKlxAAhQbpO5PmvhjpF02H/s4/+JQsfZlg2TEq
gT/7awf/aSyp+45tFzTJJGexh2Elw9dzjPQ12nPMJJlW4MhPrrgh8LRmQlmO9PKY
B+Me/zseB9L4hdmU4sLpIojNsiIXQXz7R/vznJI+y0FY02D2PDx7NsTo+fm8zZqx
ToCgWVJhfnygDIOTcEntMkeLwjvSe0bir9GcqTZef4RArZcFXKeAl1lNOddhxnh2
siZe9NPEeBtRHSA4NJcVSPs6GDLtldtalEfd93+GY/9L1IoClTeMUFAT3zB+zTIJ
EaoRpD9dv7MS1pS7NNe2AjgK71d2Ye8AKL5RfbTZujMGB5Cf9ucm80bf/Z6UFZb3
YvuMgZvej7Xt/POmUw7h9935wipiWLWY94pRHHY/VTtoaMO5kHQXWyV8xpbeMIQX
umEqGf+m8w48Ze1ZXt4QTMsH3lsZrncx7Vab8zgqz2zN2VzElRZ/TX/NQ2pTnY1n
Yr76gj9WHoL8tmbGLQ/cQemk4kXA5zEucV+L1M+ejXK7IfMBDZcbnCFIb/ROTE1q
kP1HfFbFfnXJUw7FI9o7Yyqr1FDpHAM4rzCTeZtUj6kflhTuHB/tDaKxn4PhFQ2Z
lzJcj5xsEEGKLUEyqn9QHdDwJysn63+QiGgXBR2d9oDgAdTS7Y5wICnq/9JHySrc
J7fYkCReoWg=
=+1uz
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.