Debian Bug report logs - #992775
python3.9: stores wrong path to mkdir if built on unified-/usr system

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: python3.9: stores wrong path to mkdir if built on unified-/usr system

Date: Mon, 23 Aug 2021 10:21:13 +0100

[Message part 1 (text/plain, inline)]

Source: python3.9
Version: 3.9.2-1
Severity: important
Tags: patch bookworm sid
User: reproducible-builds@lists.alioth.debian.org
Usertags: usrmerge
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

If python3.9 is built on a merged-/usr system (as created by new
installations of Debian >= 10, debootstrap --merged-usr, or installing the
usrmerge package into an existing installation) or on a system where /bin
and /usr/bin have been unified via a "symlink farm" in /bin, the path to
mkdir is recorded in installed Makefiles as "MKDIR_P = /usr/bin/mkdir -p"
instead of the canonical "MKDIR_P = /bin/mkdir -p".

This cannot currently be seen on the reproducible-builds.org infra, because
the python3.9 "build2" times out, but it can be reproduced by building
python3.9 locally.

If you have sbuild available, a way to reproduce this is:

- build twice, once with --add-depends-arch=usrmerge and once without
- both times, build with --build-path=/build/python or similar, to
  eliminate unrelated variation that involves the build path
- both times, build with DEB_BUILD_OPTIONS=noopt, to eliminate unrelated
  variation which appears to be caused by profile-guided optimization or
  by LTO
- compare built results with diffoscope
- ignore unrelated variation in usr/share/info/python3.9.info.gz

The problematic situation is if the package is *built* on a unified-/usr
system, but *used* on a non-unified-/usr system. In this situation,
/usr/bin/mkdir exists on the build system but not on the system where
python3.9 will be used, resulting in the installed Makefile being
unsuitable for the installed system.

Technical Committee resolution #978636 mandates heading towards a
transition to merged-/usr (as Ubuntu has already done), and this
will become a non-issue at the end of that transition; but variation
between merged-/usr and non-merged-/usr builds is a problem while that
transition is taking place, because it can lead to partial upgrades
behaving incorrectly. It is likely that this class of bugs will become
release-critical later in the bookworm development cycle.

Some developers advocate that instead of merged-/usr, /usr/bin and /bin
should be made equivalent via a "symlink farm" in /bin, but that would
have the same effect as merged-/usr for this particular package: the
transitional problem and the solution are the same.

The attached patch resolves this: with it applied, the package builds
more similarly with and without --add-depends-arch=usrmerge.

A side benefit of fixing this is that this change brings the package
a little closer to being reproducible (as recommended by Policy §4.15).

I suspect python3.10 has the same issue and would benefit from the same
solution, but I haven't tested that. If the attached patch seems valid,
please apply it to both.

    smcv

[0001-d-rules-Specify-canonical-path-to-mkdir.patch (text/x-diff, attachment)]

Message #10 received at 992775-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 992775-close@bugs.debian.org

Subject: Bug#992775: fixed in python3.9 3.9.7-2

Date: Fri, 03 Sep 2021 06:49:04 +0000

Source: python3.9
Source-Version: 3.9.7-2
Done: Matthias Klose <doko@debian.org>

We believe that the bug you reported is fixed in the latest version of
python3.9, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 992775@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Matthias Klose <doko@debian.org> (supplier of updated python3.9 package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Fri, 03 Sep 2021 08:18:44 +0200
Source: python3.9
Architecture: source
Version: 3.9.7-2
Distribution: unstable
Urgency: medium
Maintainer: Matthias Klose <doko@debian.org>
Changed-By: Matthias Klose <doko@debian.org>
Closes: 989000 991110 992091 992669 992775 993413
Changes:
 python3.9 (3.9.7-2) unstable; urgency=medium
 .
   * Fix CFLAGS in the python3.x-config scripts. Closes: #992669, #989000.
   * Support arc-linux-gnu in the multiarch header. Closes: #992091.
   * Don't run the test_concurrent_futures test on alpha and sparc64.
     Closes: #991110.
   * Fix path to mkdir on a unified-/usr system. Closes: #992775.
   * Use command -v instead of which in shell scripts. Closes: #993413.
Checksums-Sha1:
 17ef1d15102029c66744acb83eac93c1bfdb4165 3493 python3.9_3.9.7-2.dsc
 f011a4231b2d6d51af4f70f5e838491a9c6ce38c 212164 python3.9_3.9.7-2.debian.tar.xz
 bb3310ec52ac37947e13658074daa864dda59b15 10182 python3.9_3.9.7-2_source.buildinfo
Checksums-Sha256:
 7b14f5d79bd3a33ff2ffac479655416b139222a18e7e93b6afe6b69b4821ec4b 3493 python3.9_3.9.7-2.dsc
 80e2be75e2b1e7553a828a8c45d27852e4b054375875b7408e99669e75e3c937 212164 python3.9_3.9.7-2.debian.tar.xz
 7e9345550c51e9a6a7bac60ef49831fafa0422e7bcbaf74915f10375aca5c741 10182 python3.9_3.9.7-2_source.buildinfo
Files:
 15f1c468a2eb2d25e6157efaf69eb8b0 3493 python optional python3.9_3.9.7-2.dsc
 751091f441daa0a73fbaddff5a9a2875 212164 python optional python3.9_3.9.7-2.debian.tar.xz
 8a40b4a4126fd3e8fff876403d3d2803 10182 python optional python3.9_3.9.7-2_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQJEBAEBCAAuFiEE1WVxuIqLuvFAv2PWvX6qYHePpvUFAmExwG8QHGRva29AZGVi
aWFuLm9yZwAKCRC9fqpgd4+m9QGnEACjGY7FIq519X9/41eagYOofuSWhuvEW5hn
5MaS+TE09oKuUoJEXZ8xqwxI+cyUOW3jtNNC0kfqWuoTPrvMWNbhZpeLZIIYCn8Q
+dHOsc8zmjl0U2uyFjF2EIWk9JoE/RJswjRueeP1lAusZ2lY6fpLXkihKldizOmj
1gNlEDSxjDPeKFvD625ijN92yZBi3qM4yqxZH1MdUPRLYEkVRxHv8FOjvUhebxhp
EvIOVCQEJD4pi5KzrAsDYaGpaw6dfuqDmOyDw2l7WR1oev6DfjGSJh18O5kswBkb
aVCLt2Zpm24HTyw0KAkX7gh8WO2JKN1nNMqWDSsN8w5XM7U988yLPFL6XhpARN3V
bgL5QmiBNtjQwA1NiszzDrw+CHkwMdnnB1LZG+hFZpsyOrNPHGGBuqKE+ZoUfcz4
w+tVzoZgvh5eC2CVVCTikvQQjeOAi8Hqs/WM8T4n6gBY2GO21Pxhm9C8OlbcIk7Y
TCc7uCqa8ClRoFq/S+P5F6bDEQo9RYwppWTJVtixS0xOTPfaab7pPa4HIQ4+7VLq
T4g4SIPUIex5BgH7nNlF8M/I7xaXGRepzyM+yFgcUAU2InvRvefbqG7+pPOlDG8v
p9mzVV7sU1NFf0Ocxhf8N7DjZ5G9CdFW+kkUo9AVvMYO0VAESKSOSYHkNC/Ryq5B
NWKnv2vH1Q==
=QBOs
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.