Debian Bug report logs - #786913
ocamlc -custom creates random intermediate filenames

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

To: submit@bugs.debian.org

Subject: ocaml creates random intermediate filenames, makes built packages non-reproducible

Date: Tue, 26 May 2015 13:11:06 -0400

[Message part 1 (text/plain, inline)]

Package: ocaml
Version: 4.01.0-5
User: reproducible-builds@lists.alioth.debian.org
Usertags: toolchain randomness
Control: affects -1 nss-passwords

ocaml makes some debug builds non-reproducible by embedding some entropy
in the debug symbols.  Below is an attempt at a diagnosis and sketch of
a couple of possible fixes.

debian/patches/0008-Embed-bytecode-in-C-object-when-using-custom.patch
creates an arbirtrarily-named tempfile during some builds:

 c_file = Filename.temp_file "camlobj" ".c" in

This tempfile name ends up being included in the the debugging symbols
as an explicit name.  The debugging symbols then end up having a
different checksum based on the random name of the file.

This can be seen in:

  https://reproducible.debian.net/dbd/unstable/amd64/nss-passwords_0.2-1.debbindiff.html

where the only difference between the two builds is the random filename
in the debug symbols, and the checksum in the .gnu_debuglink section of
the resultant binary.

I'm not sure exactly how to best resolve this, but i see two general
approaches that might be useful:

 * derive a deterministic filename from the build-state or contents of
   the generated code, and use that filename instead of a random
   tmpfile.  (this has the disadvantage that it might be possible for
   this to collide with an existing file, which would produce a new
   possible error state for the build, but could be made pretty unlikely
   in the commen case)

 * create a temporary directory using the standard random naming scheme,
   and use a known filename within that directory.  (this would require
   that the debug symbols not embed the name of the directory)

Thanks for maintaining ocaml in debian!

Regards,

   --dkg

[signature.asc (application/pgp-signature, inline)]

Message #18 received at 786913@bugs.debian.org (full text, mbox, reply):

From: Stéphane Glondu <glondu@debian.org>

To: Valentin Lorentz <progval@progval.net>, 794583@bugs.debian.org

Cc: 786913@bugs.debian.org, 795784@bugs.debian.org, 796336@bugs.debian.org

Subject: Re: Bug#794583: ocaml: Allow setting arbitrary RNG seed in ocamlopt

Date: Tue, 25 Aug 2015 16:00:32 +0200

Le 04/08/2015 18:41, Valentin Lorentz a écrit :
> While working on the “reproducible builds” effort [1], we have noticed
> that ocamlopt relies on temporary files whose names are generated
> randomly and are part of the output files' symbols.

See #795784, #796336 and #786913.

> Therefore, we need a way to make these names determinist. [...]

I don't agree with this conclusion; I'd rather use a way to not record
those random names in the first place, or set them to some sane value
without messing with file names.

GCC also uses temporary files whose names are generated randomly (this
can be seen with the -v option). But it arranges for these random names
to not appear in compiled objects.

For example, with assembly files, the name of the "source" file (which
is then recorded in compiled objects) can be given with a ".file"
directive. gcc adds them to its assembly files. And adding such
directives to assembly files generated by ocamlopt solves #795784 and
#796336.

For #786913, temporary files are C files. Ideally, a ".file" counterpart
should exist for C files (I thought of "#line" cpp directives, but they
don't work... maybe we should make them work?). However, I've found a
way to tell gcc to not record the file name, using stdin: gcc -x c -c -o
foo.o < foo.c. I would very much prefer a ".file"-like directive, though.

> [...] For instance,
> reading an environment variable in the main function of ocamlopt
> (driver/optmain.ml) and calling “Random.seed 0” if it is set would be
> perfect.
> Using OCAMLPARAM (driver/compenv.ml) would work as well.

I am not thrilled by this proposition. Filename.temp_file is the
equivalent of mkstemp, and mkstemp doesn't have this "feature".


Cheers,

-- 
Stéphane

Send a report that this bug log contains spam.