Debian Bug report logs - #901307
sphinx-gallery: please make the build mostly reproducible

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: submit@bugs.debian.org

Subject: sphinx-gallery: please make the build mostly reproducible

Date: Mon, 11 Jun 2018 09:16:09 +0100

[Message part 1 (text/plain, inline)]

Source: sphinx-gallery
Version: 0.1.13-2
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: timestamps
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

Hi,

Whilst working on the Reproducible Builds effort [0], we noticed
that sphinx-gallery could not be built reproducibly.

Patch attached that removes the "Total·​running·​time·​of·​the·​script"
messages from the documentation. (There is some filesystem-related
unreproduciblity that remains, alas.)

 [0] https://reproducible-builds.org/


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

[sphinx-gallery.diff.txt (text/plain, attachment)]

Message #10 received at 901307@bugs.debian.org (full text, mbox, reply):

From: Chris Lamb <lamby@debian.org>

To: 901307@bugs.debian.org

Subject: Re: sphinx-gallery: please make the build mostly reproducible

Date: Mon, 11 Jun 2018 09:23:52 +0100

forwarded 901307 https://github.com/sphinx-gallery/sphinx-gallery/pull/390
thanks

I've forwarded this upstream here:

  https://github.com/sphinx-gallery/sphinx-gallery/pull/390


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #17 received at 901307-submitter@bugs.debian.org (full text, mbox, reply):

From: Sandro Tosi <noreply@salsa.debian.org>

To: 901307-submitter@bugs.debian.org

Subject: Bug#901307 marked as pending in sphinx-gallery

Date: Sat, 07 Mar 2020 04:20:08 +0000

Control: tag -1 pending

Hello,

Bug #901307 in sphinx-gallery reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/python-team/modules/sphinx-gallery/-/commit/2d151361a996c1caa1b2f8606da7a9cf84b39aa8

------------------------------------------------------------------------
New upstream release; Closes: #901307
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/901307

Message #24 received at 901307-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 901307-close@bugs.debian.org

Subject: Bug#901307: fixed in sphinx-gallery 0.5.0-1

Date: Sat, 07 Mar 2020 04:49:50 +0000

Source: sphinx-gallery
Source-Version: 0.5.0-1
Done: Sandro Tosi <morph@debian.org>

We believe that the bug you reported is fixed in the latest version of
sphinx-gallery, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 901307@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Sandro Tosi <morph@debian.org> (supplier of updated sphinx-gallery package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Fri, 06 Mar 2020 22:20:36 -0500
Source: sphinx-gallery
Architecture: source
Version: 0.5.0-1
Distribution: unstable
Urgency: medium
Maintainer: Sandro Tosi <morph@debian.org>
Changed-By: Sandro Tosi <morph@debian.org>
Closes: 901307
Changes:
 sphinx-gallery (0.5.0-1) unstable; urgency=medium
 .
   * New upstream release; Closes: #901307
   * debian/copyright
     - extend packaging copyright years
   * debian/patches
     - drop all patches, merged upstream
   * debian/control
     - add sphinx-rtd-theme, memory-profiler, ipython to b-d, needed by doc
     - bump Standards-Version to 4.5.0 (no changes needed)
   * debian/patches/doc-no-mayavi.patch
     - remove mayavi references, since we're not building those examples
   * Use local intersphinx mapping files, if available
Checksums-Sha1:
 8be5bc952ea438fd18ff8b9ec3aa2a6bb2404ebf 2476 sphinx-gallery_0.5.0-1.dsc
 1ec61a1d0bda426071b49b014e55957a846f0f19 367192 sphinx-gallery_0.5.0.orig.tar.xz
 d83a9ba2cb7366add6d54990e10d4f800f57113b 3748 sphinx-gallery_0.5.0-1.debian.tar.xz
 c67ce99fcaf2c372456c40411b91041e09fa7ae5 11291 sphinx-gallery_0.5.0-1_amd64.buildinfo
Checksums-Sha256:
 033aff0fc8b61e0f4d83f5ac6fac84bad0d4a3d59ffefa9132acf4ae0652d5d2 2476 sphinx-gallery_0.5.0-1.dsc
 445f7c6547f5ae7a6f1203a6ca7d9c24d0e36726022c10c4a274b65e9d2c5b57 367192 sphinx-gallery_0.5.0.orig.tar.xz
 40c4e73f1223962ed51647c8d39199b4d63b782496dc40a1fba7235a0fcde1f9 3748 sphinx-gallery_0.5.0-1.debian.tar.xz
 d1cf04e94211498f9d766503fe4e805635002ad242fc1ee6bfa22a95a76d89a2 11291 sphinx-gallery_0.5.0-1_amd64.buildinfo
Files:
 84c4c4d90c8332e0bfd6c47c8816b913 2476 python optional sphinx-gallery_0.5.0-1.dsc
 f1838227217ae3c63a3596370c3f5a37 367192 python optional sphinx-gallery_0.5.0.orig.tar.xz
 c0ed288cc19cb6adefb270f0a57ae603 3748 python optional sphinx-gallery_0.5.0-1.debian.tar.xz
 4d5cc6a19925218a05c41d1bcf16fef1 11291 python optional sphinx-gallery_0.5.0-1_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEufrTGSrz5KUwnZ05h588mTgBqU8FAl5jIVMACgkQh588mTgB
qU80ihAAobzHcrskgkNE/lGp//YY6Eli+nXnR2JsVMHKdoFdrRxUFgtvXoumdAE8
/zTWZZVyOnXQNTc9B+oYBoyesSU5Io+kEr8S5cNyIKUGH9aR5zuxa7a6+NvsDMq6
zMaDygjhMzVygaIofqe+SCmy+B0I5R/yu27Jk/wmP7YoJp9yoboz+QMKou40Vcr8
X64wwh8nbcQ5iIVjv241AYmKZumF9HsiG5ktwsNegrpK1rUHokxZpAL1wtOvSm8u
ecitRfny/ggriSE4bxiq3IqAuX4aOqVBiJasSiEsDVD+9i5eTrwkuX1jBtnIe2G4
aNtczq7U1ttDjhjQDNzZYmYEI3KAoRkx0w64Qy/hUImBKZ7y4cVTO46cYXAhUpG6
IG1CAuH0CA5YDxsh8TyGycKsGp2hzjsbeVFDN8FXVyM3aR60gz/WZ6pA+TkQgboX
c6leEWAjTkki4UT6Z9ecIbFnDR0YldPd63A5q/P30cFpIl9MUd8jTpYl3cEv5VoT
62HVINPZ2UQ4vX82BO5aNXcBWh6M6ibe9q7+dnKF2mBDJc5VHCJHqeVSI2LErso+
Jg+IgmMMwAo/J9CUWqIA3MuRzY12v+hhlE+rLj8lvRK3Bumbu61nVCSp6q0ZzVlj
ndlKxGh0ZX6Ons7PDH/liCQqi4xMncIiadhpSsguoIm2Q8oajkE=
=KLKc
-----END PGP SIGNATURE-----

Message #33 received at 901307@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 901307@bugs.debian.org

Subject: Re: sphinx-gallery: please make the build mostly reproducible

Date: Wed, 05 Aug 2020 11:07:56 +0100

reopen 901307
found 901307 0.7.0-1
thanks

Hi,

Alas, I think this issue has come back. lmfit-py 1.0.1-1 in unstable
right now is being built with sphinx-gallery 0.7.0-1 but I am seeing:

   -   **Total running time of the script:** ( 0 minutes  0.443 seconds)
   +   **Total running time of the script:** ( 0 minutes  0.361 seconds)

… and similar. The full diffoscope output of lmfit-py is linked below,
although note that it has differences that are not due to
sphinx-gallery:

  https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/lmfit-py.html


Regards,

--
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #44 received at 901307@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: 901307@bugs.debian.org, reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#901307: sphinx-gallery: please make the build mostly reproducible

Date: Wed, 04 Nov 2020 13:04:03 -0000

Chris Lamb wrote:

> [..]

Friendly ping on this?


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org / chris-lamb.co.uk
       `-

Message #49 received at 901307@bugs.debian.org (full text, mbox, reply):

From: Sandro Tosi <morph@debian.org>

To: Chris Lamb <lamby@debian.org>, 901307@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#901307: sphinx-gallery: please make the build mostly reproducible

Date: Sun, 10 Oct 2021 23:12:41 -0400

> Friendly ping on this?

this was supposed to be fixed when i uploaded that version. Anyhow,
i've just uploaded 0.10.0 to unstable, can you  check that version
out? if it's still FTBR maybe give a ping upstream about it?

Thanks,
-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

Message #54 received at 901307@bugs.debian.org (full text, mbox, reply):

From: "Chris Lamb" <lamby@debian.org>

To: "Sandro Tosi" <morph@debian.org>, 901307@bugs.debian.org

Cc: reproducible-bugs@lists.alioth.debian.org

Subject: Re: Bug#901307: sphinx-gallery: please make the build mostly reproducible

Date: Mon, 11 Oct 2021 15:04:11 +0100

Hi Sandro,

>> Friendly ping on this?
>
> this was supposed to be fixed when i uploaded that version. Anyhow,
> i've just uploaded 0.10.0 to unstable, can you check that version
> out?

There are a large number of distinct issues here which will require
different solutions, not all of them involving upstream.


## 1. Memory usage

sphinx-gallery includes memory usage in generated documentation. This
can be quite easily fixed with the following patch, which is likely
very amenable to upstream given that it's merely an extension of
varying the configuration based on the presence of SOURCE_DATE_EPOCH:

    --- sphinx-gallery-0.8.2.orig/doc/conf.py
    +++ sphinx-gallery-0.8.2/doc/conf.py
    @@ -335,8 +335,10 @@ try:
     except ImportError:
         pass

    +show_memory = True
     min_reported_time = 0
     if 'SOURCE_DATE_EPOCH' in os.environ:
    +    show_memory = False
         min_reported_time = sys.maxint if sys.version_info[0] == 2 else sys.maxsize

     sphinx_gallery_conf = {
    @@ -362,7 +364,7 @@ sphinx_gallery_conf = {
                    'notebooks_dir': 'notebooks',
                    'use_jupyter_lab': True,
                    },
    -    'show_memory': True,
    +    'show_memory': show_memory,
         'junit': os.path.join('sphinx-gallery', 'junit-results.xml'),
         # capture raw HTML or, if not present, __repr__ of last expression in
         # each code block


##. 2. Execution times

Secondly, there are several instances of "total execution time for
<filename>" being included in the output. I'm not quite sure how to
fix this as simply doing something like:

    --- sphinx-gallery-0.8.2.orig/sphinx_gallery/gen_gallery.py
    +++ sphinx-gallery-0.8.2/sphinx_gallery/gen_gallery.py
    @@ -538,6 +538,9 @@ def write_computation_times(gallery_conf
         with codecs.open(os.path.join(target_dir, 'sg_execution_times.rst'), 'w',
                          encoding='utf-8') as fid:
             fid.write(SPHX_GLR_COMP_TIMES.format(new_ref))
    +        if 'SOURCE_DATE_EPOCH' in os.environ:
    +            fid.write('Not included.');
    +            return
             fid.write('**{0}** total execution time for **{1}** files:\n\n'
                       .format(_sec_to_readable(total_time), target_dir_clean))
             lines, lens = _format_for_writing(costs, target_dir_clean)

... results in a kinda weird "sg_execution_times" manual page with no
content and is linked to elsewhere in the documentation. Not sure how
to either a) do a conditional paragraph in Sphinx documentation or,
preferably, b) what to include in this 'null' page beyond a more
verbose version of "Not included."


### 3. sys.argv included in example

Literal contents of sys.argv included in example which then ends up in
the documentation. This therefore includes the full build path.

    --- sphinx-gallery-0.8.2.orig/examples/plot_7_sys_argv.py
    +++ sphinx-gallery-0.8.2/examples/plot_7_sys_argv.py
    @@ -13,10 +13,11 @@ This behavior can be changed by using th
     """

     import argparse
    +import os
     import sys

     parser = argparse.ArgumentParser(description='Toy parser')
     parser.add_argument('--option', default='default',
                         help='a dummy optional argument')
    -print('sys.argv:', sys.argv)
    +print('sys.argv:', [x.replace(os.path.dirname(os.getcwd()), "/path/to") for x in sys.argv])
     print('parsed args:', parser.parse_args())


### 4. Package contains Python bytecode (.pyc)

This probably shouldn't be happening from a Debian packaging PoV:

  /usr/share/doc/python-sphinx-gallery-doc/examples/__pycache__/local_module.cpython-39.pyc


### 5. junit-results.xml includes timing info

This file includes timing info:

  /usr/share/doc/python-sphinx-gallery-doc/html/sphinx-gallery/junit-results.xml

The value of distributing this file seems negligible (if not zero), so
it's likely this can simply be dropped from the package.


### 6. Memory references in MathJax output

eg. "[<Line3D object at 0xabcdef00]"

    --- sphinx-gallery-0.8.2.orig/examples/plot_3_capture_repr.py
    +++ sphinx-gallery-0.8.2/examples/plot_3_capture_repr.py
    @@ -86,7 +86,7 @@ statsmodels.iolib.table.SimpleTable(np.z

     import matplotlib.pyplot as plt

    -plt.plot([1,2,3])
    +[type(x) for x in plt.plot([1,2,3])]

     #%%
     # To avoid capturing the text representation, you can assign the last Matplotlib


Okay, need to do work on something else for a bit, and that's probably
enough for now...


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby@debian.org 🍥 chris-lamb.co.uk
       `-

Send a report that this bug log contains spam.