Debian Bug report logs - #850284
reprotest: add a mode to automatically attempt to detect which variations cause unreproducibility

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Ximin Luo <infinity0@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: reprotest: add a mode to automatically attempt to detect which variations cause unreproducibility

Date: Thu, 05 Jan 2017 18:07:58 +0100

Package: reprotest
Version: 0.4
Severity: wishlist

Dear Maintainer,

One idea that was brought up at RWS2 in Berlin (I forgot who came up with it
now, sorry) is to add a mode to automatically attempt to detect which
variations cause a package to be unreproducible.

For example, given variations {a, b, c}, if varying none of these results in a
reproducible build, but varying all of them makes it unreproducible, then
reprotest could automatically try to figure out which combinations of {a, b, c}
"cause" the unreproducibility.

Note that this is not as simple as a "bisect" algorithm which assumes that the
items to be tested are in some linear order. In fact, it's unclear if we can
make the algorithm (using the above example) more efficient than the brute
force method of "test a, test b, test c".

X

-- System Information:
Debian Release: stretch/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'testing-debug'), (300, 'unstable'), (200, 'experimental'), (1, 'experimental-debug')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.8.0-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.utf8, LC_CTYPE=en_GB.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages reprotest depends on:
ii  apt-utils              1.4~beta2
ii  diffoscope             63
ii  libdpkg-perl           1.18.18
ii  procps                 2:3.3.12-3
ii  python3-debian         0.1.29
ii  python3-pkg-resources  32.0.0-1
pn  python3:any            <none>

Versions of packages reprotest recommends:
ii  disorderfs   0.5.1-1
ii  locales-all  2.24-8

Versions of packages reprotest suggests:
ii  autodep8     0.8
ii  qemu-system  1:2.7+dfsg-3+b1
ii  qemu-utils   1:2.7+dfsg-3+b1
ii  schroot      1.6.10-2+b2

-- no debconf information

Message #10 received at 850284@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: 850284@bugs.debian.org

Subject: Re: Bug#850284: reprotest: add a mode to automatically attempt to detect which variations cause unreproducibility

Date: Fri, 6 Jan 2017 16:19:20 +0000

[Message part 1 (text/plain, inline)]

On Thu, Jan 05, 2017 at 06:07:58PM +0100, Ximin Luo wrote:
> One idea that was brought up at RWS2 in Berlin (I forgot who came up with it
> now, sorry) is to add a mode to automatically attempt to detect which
> variations cause a package to be unreproducible.
 
kudos for that idea, again! :)

> Note that this is not as simple as a "bisect" algorithm which assumes that the
> items to be tested are in some linear order. In fact, it's unclear if we can
> make the algorithm (using the above example) more efficient than the brute
> force method of "test a, test b, test c".

I'm not fully sure how to take this into consideration, but some
variations are more likely to cause unreproducible builds than others,
IOW the ordering of the test shouldnt be random, but rather follow such
(assumed) probability.


-- 
cheers,
	Holger

[signature.asc (application/pgp-signature, inline)]

Message #15 received at 850284-done@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: Ximin Luo <infinity0@debian.org>, 850284-done@bugs.debian.org

Subject: Re: Bug#850284: reprotest: add a mode to automatically attempt to detect which variations cause unreproducibility

Date: Tue, 06 Oct 2020 12:06:29 -0700

[Message part 1 (text/plain, inline)]

Version: 0.7.1

On 2017-01-05, Ximin Luo wrote:
> One idea that was brought up at RWS2 in Berlin (I forgot who came up with it
> now, sorry) is to add a mode to automatically attempt to detect which
> variations cause a package to be unreproducible.

This was implemented with the --auto-build option:

    group1_0.add_argument('--auto-build', default=False, action='store_true',
        help='Automatically perform builds to try to determine which specific '
        'variations cause unreproducibility, potentially up to and including '
        'the ones specified by --variations and --vary. Conflicts with '
        '--extra-build.')

> For example, given variations {a, b, c}, if varying none of these results in a
> reproducible build, but varying all of them makes it unreproducible, then
> reprotest could automatically try to figure out which combinations of {a, b, c}
> "cause" the unreproducibility.

It doesn't figure out combinations, e.g. if a+b cause a issue that only
individually running a or only individually running b individually would
not trigger, but I'm guessing such complicated issues combinations are
pretty rare.

If you think this is important enough or I missed something in
understanding the issue, please re-open.


> Note that this is not as simple as a "bisect" algorithm which assumes that the
> items to be tested are in some linear order. In fact, it's unclear if we can
> make the algorithm (using the above example) more efficient than the brute
> force method of "test a, test b, test c".

I think the --auto-build option essentially implements the brute-force option.


live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.