Debian Bug report logs - #985553
dpkg-buildflags: RFC use gcc .spec files for fixfilepath/fixdebugpath

Package: dpkg-dev; Maintainer for dpkg-dev is Dpkg Developers <debian-dpkg@lists.debian.org>; Source for dpkg-dev is src:dpkg (PTS, buildd, popcon).

Reported by: Vagrant Cascadian <vagrant@reproducible-builds.org>

Date: Fri, 19 Mar 2021 18:36:01 UTC

Severity: wishlist

Tags: moreinfo, patch

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, reproducible-bugs@lists.alioth.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#985553; Package dpkg. (Fri, 19 Mar 2021 18:36:03 GMT) (full text, mbox, link).


Acknowledgement sent to Vagrant Cascadian <vagrant@reproducible-builds.org>:
New Bug report received and forwarded. Copy sent to reproducible-bugs@lists.alioth.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>. (Fri, 19 Mar 2021 18:36:03 GMT) (full text, mbox, link).


Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: dpkg: RFC use gcc .spec files for fixfilepath/fixdebugpath
Date: Fri, 19 Mar 2021 11:32:52 -0700
[Message part 1 (text/plain, inline)]
Package: dpkg
Severity: normal
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: buildpath
X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

While the reproducible=+fixfilepath and reproducible=+fixdebugpath are
very helpful in stripping out the build path from binaries, many
binaries still end up including the build path because they also embed
the compiler flags (e.g. -ffile-prefix-map=/path/to/build-XYZ=.):

  https://tests.reproducible-builds.org/debian/issues/unstable/records_build_flags_issue.html

At the moment, we have identified over 150 packages in Debian affected
by this issue, though there are probably more that have not yet been
identified. Some packages may work around this issue by sanitizing or
stripping -ffile-prefix-map and -fdebug-prefix-map, but handling this on
a per-package basis is a bit of a whack-a-mole approach.


The attached patch attempts to switch fixfilepath and fixdebugpath to
use .spec files with the DEB_BUILD_PATH environment variable, which is
currently used internally for the fixfilepath and fixdebugpath
features.

Note that it is only a partial patch; it doesn't export DEB_BUILD_PATH
in dpkg-buildpackage and/or dpkg-buildflags; help as to the best place
to implement that would be appreciated!

Instead of changing fixfilepath/fixdebugpath, it might also be better to
add new options instead, as this is a significant change of the
implementation.


Interested to hear your thoughts everyone!


live well,
  vagrant

[0001-Implement-reproducible-fixfilepath-fixdebugpath-as-s.patch (text/x-diff, inline)]
From 2f2985a12673866a341551399a5461fe8d7e617b Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@reproducible-builds.org>
Date: Sun, 14 Mar 2021 00:19:56 -0800
Subject: [PATCH] Implement reproducible=(fixfilepath|fixdebugpath) as spec
 files.

This will avoid embedding the build path in the compiler flags:

  https://tests.reproducible-builds.org/debian/issues/unstable/records_build_flags_issue.html
---
 Makefile.am                   | 2 ++
 data/fixdebugpath.specs       | 2 ++
 data/fixfilepath.specs        | 2 ++
 scripts/Dpkg/Vendor/Debian.pm | 4 ++--
 4 files changed, 8 insertions(+), 2 deletions(-)
 create mode 100644 data/fixdebugpath.specs
 create mode 100644 data/fixfilepath.specs

diff --git a/Makefile.am b/Makefile.am
index 180562e19..a4f70c420 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -38,6 +38,8 @@ dist_pkgdata_DATA = \
 	data/pie-link.specs \
 	data/no-pie-compile.specs \
 	data/no-pie-link.specs \
+	data/fixfilepath.specs \
+	data/fixdebugpath.specs \
 	data/cputable \
 	data/ostable \
 	data/abitable \
diff --git a/data/fixdebugpath.specs b/data/fixdebugpath.specs
new file mode 100644
index 000000000..e1cc16b0a
--- /dev/null
+++ b/data/fixdebugpath.specs
@@ -0,0 +1,2 @@
+*self_spec:
++ -fdebug-prefix-map=%:getenv(DEB_BUILD_PATH =.)
diff --git a/data/fixfilepath.specs b/data/fixfilepath.specs
new file mode 100644
index 000000000..1f8561575
--- /dev/null
+++ b/data/fixfilepath.specs
@@ -0,0 +1,2 @@
+*self_spec:
++ -ffile-prefix-map=%:getenv(DEB_BUILD_PATH =.)
diff --git a/scripts/Dpkg/Vendor/Debian.pm b/scripts/Dpkg/Vendor/Debian.pm
index 06770a235..28cc3c507 100644
--- a/scripts/Dpkg/Vendor/Debian.pm
+++ b/scripts/Dpkg/Vendor/Debian.pm
@@ -256,9 +256,9 @@ sub _add_build_flags {
         # -ffile-prefix-map is a superset of -fdebug-prefix-map, prefer it
         # if both are set.
         if ($use_feature{reproducible}{fixfilepath}) {
-            $map = '-ffile-prefix-map=' . $build_path . '=.';
+            $map = "-specs=$Dpkg::DATADIR/fixfilepath.specs";
         } else {
-            $map = '-fdebug-prefix-map=' . $build_path . '=.';
+            $map = "-specs=$Dpkg::DATADIR/fixdebugpath.specs";
         }
 
         $flags->append('CFLAGS', $map);
-- 
2.30.2

[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#985553; Package dpkg. (Mon, 23 Aug 2021 21:15:02 GMT) (full text, mbox, link).


Acknowledgement sent to Guillem Jover <guillem@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Mon, 23 Aug 2021 21:15:02 GMT) (full text, mbox, link).


Message #10 received at 985553@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>
To: Vagrant Cascadian <vagrant@reproducible-builds.org>, 985553@bugs.debian.org
Subject: Re: Bug#985553: dpkg: RFC use gcc .spec files for fixfilepath/fixdebugpath
Date: Mon, 23 Aug 2021 23:11:01 +0200
Hi!

On Fri, 2021-03-19 at 11:32:52 -0700, Vagrant Cascadian wrote:
> Package: dpkg
> Severity: normal
> Tags: patch
> User: reproducible-builds@lists.alioth.debian.org
> Usertags: buildpath
> X-Debbugs-Cc: reproducible-bugs@lists.alioth.debian.org

> While the reproducible=+fixfilepath and reproducible=+fixdebugpath are
> very helpful in stripping out the build path from binaries, many
> binaries still end up including the build path because they also embed
> the compiler flags (e.g. -ffile-prefix-map=/path/to/build-XYZ=.):
> 
>   https://tests.reproducible-builds.org/debian/issues/unstable/records_build_flags_issue.html
> 
> At the moment, we have identified over 150 packages in Debian affected
> by this issue, though there are probably more that have not yet been
> identified. Some packages may work around this issue by sanitizing or
> stripping -ffile-prefix-map and -fdebug-prefix-map, but handling this on
> a per-package basis is a bit of a whack-a-mole approach.
> 
> 
> The attached patch attempts to switch fixfilepath and fixdebugpath to
> use .spec files with the DEB_BUILD_PATH environment variable, which is
> currently used internally for the fixfilepath and fixdebugpath
> features.

Ah I like this direction, yes!

> Note that it is only a partial patch; it doesn't export DEB_BUILD_PATH
> in dpkg-buildpackage and/or dpkg-buildflags; help as to the best place
> to implement that would be appreciated!

The ideal place to export such variable would be from
dpkg-buildpackage, but sadly as long as that's not the canonical entry
point for building packages, I'm seeing only doing that if the
implementation can avoid assuming the variable is set and shows no
regressions in that case.

> Instead of changing fixfilepath/fixdebugpath, it might also be better to
> add new options instead, as this is a significant change of the
> implementation.

If we'd be worried that the new behavior can break stuff, then that
might make sense, but that would either imply that we need to manually
switch current packages to explicitly use the new ones, or we need
to flip the new ones as the default and disable the old ones, in case
some package needs the old behavior. But that all seems a bit unnecessary
to me?

> Interested to hear your thoughts everyone!


> From 2f2985a12673866a341551399a5461fe8d7e617b Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
> Date: Sun, 14 Mar 2021 00:19:56 -0800
> Subject: [PATCH] Implement reproducible=(fixfilepath|fixdebugpath) as spec
>  files.

> diff --git a/data/fixdebugpath.specs b/data/fixdebugpath.specs
> new file mode 100644
> index 000000000..e1cc16b0a
> --- /dev/null
> +++ b/data/fixdebugpath.specs
> @@ -0,0 +1,2 @@
> +*self_spec:
> ++ -fdebug-prefix-map=%:getenv(DEB_BUILD_PATH =.)

I'm not sure the equal sign is correct here? Also there's a significant
and potentially problematic behavior change here. The perl code
currently uses Cwd::getcwd(), which will return an absolute pathname,
which will be stable regardless of where the option is being used.
Instead when passing «.» that suddenly becomes relative to the call
site of each compiler invocation (not even of the dpkg-buildflags
call. :/

> diff --git a/data/fixfilepath.specs b/data/fixfilepath.specs
> new file mode 100644
> index 000000000..1f8561575
> --- /dev/null
> +++ b/data/fixfilepath.specs
> @@ -0,0 +1,2 @@
> +*self_spec:
> ++ -ffile-prefix-map=%:getenv(DEB_BUILD_PATH =.)

Ditto.

Thanks,
Guillem



Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#985553; Package dpkg. (Tue, 24 Aug 2021 00:09:02 GMT) (full text, mbox, link).


Acknowledgement sent to Vagrant Cascadian <vagrant@reproducible-builds.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Tue, 24 Aug 2021 00:09:02 GMT) (full text, mbox, link).


Message #15 received at 985553@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>
To: Guillem Jover <guillem@debian.org>, 985553@bugs.debian.org
Subject: Re: Bug#985553: dpkg: RFC use gcc .spec files for fixfilepath/fixdebugpath
Date: Mon, 23 Aug 2021 17:07:13 -0700
[Message part 1 (text/plain, inline)]
On 2021-08-23, Guillem Jover wrote:
> On Fri, 2021-03-19 at 11:32:52 -0700, Vagrant Cascadian wrote:
>> While the reproducible=+fixfilepath and reproducible=+fixdebugpath are
>> very helpful in stripping out the build path from binaries, many
>> binaries still end up including the build path because they also embed
>> the compiler flags (e.g. -ffile-prefix-map=/path/to/build-XYZ=.):
...
>> The attached patch attempts to switch fixfilepath and fixdebugpath to
>> use .spec files with the DEB_BUILD_PATH environment variable, which is
>> currently used internally for the fixfilepath and fixdebugpath
>> features.
>
> Ah I like this direction, yes!

Great!


>> Note that it is only a partial patch; it doesn't export DEB_BUILD_PATH
>> in dpkg-buildpackage and/or dpkg-buildflags; help as to the best place
>> to implement that would be appreciated!
>
> The ideal place to export such variable would be from
> dpkg-buildpackage, but sadly as long as that's not the canonical entry
> point for building packages, I'm seeing only doing that if the
> implementation can avoid assuming the variable is set and shows no
> regressions in that case.

I noticed SOURCE_DATE_EPOCH was defined in:

  /usr/share/dpkg/pkg-info.mk:SOURCE_DATE_EPOCH ?= $(call dpkg_late_eval,SOURCE_DATE_EPOCH,dpkg-parsechangelog -STimestamp)

Would it make sense to do the same for DEB_BUILD_PATH? Or would CURDIR
not be the right value at that time? Are there cases where
SOURCE_DATE_EPOCH isn't defined when building packages (e.g. fakeroot
debian/rules build). From a reproducible builds perspective, it seems
SOURCE_DATE_EPOCH is working well enough, and defining DEB_BUILD_PATH in
the same place should be good enough too.


>> Instead of changing fixfilepath/fixdebugpath, it might also be better to
>> add new options instead, as this is a significant change of the
>> implementation.
>
> If we'd be worried that the new behavior can break stuff, then that
> might make sense, but that would either imply that we need to manually
> switch current packages to explicitly use the new ones, or we need
> to flip the new ones as the default and disable the old ones, in case
> some package needs the old behavior. But that all seems a bit unnecessary
> to me?

Maybe so. *If* we do it well, there should be no real chance of
breakage!  :)


>> From 2f2985a12673866a341551399a5461fe8d7e617b Mon Sep 17 00:00:00 2001
>> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
>> Date: Sun, 14 Mar 2021 00:19:56 -0800
>> Subject: [PATCH] Implement reproducible=(fixfilepath|fixdebugpath) as spec
>>  files.
>
>> diff --git a/data/fixdebugpath.specs b/data/fixdebugpath.specs
>> new file mode 100644
>> index 000000000..e1cc16b0a
>> --- /dev/null
>> +++ b/data/fixdebugpath.specs
>> @@ -0,0 +1,2 @@
>> +*self_spec:
>> ++ -fdebug-prefix-map=%:getenv(DEB_BUILD_PATH =.)
>
> I'm not sure the equal sign is correct here?

I'm not sure what's wrong with it ...

Currently fixdebugpath would set something like:

  -fdebug-prefix-map=/absolute/path/to/build/dir=.

This gets the value of DEB_BUILD_PATH from an environment variable
instead.

Or is there something I'm not grasping with how .spec files work?


> Also there's a significant and potentially problematic behavior change
> here. The perl code currently uses Cwd::getcwd(), which will return an
> absolute pathname, which will be stable regardless of where the option
> is being used.  Instead when passing «.» that suddenly becomes
> relative to the call site of each compiler invocation (not even of the
> dpkg-buildflags call. :/

I don't *think* defining this from a .spec file really changes that
behavior, as long as you set the DEB_BUILD_PATH environment variable
only once per build... it should effectively be a fixed value.

When I tested this, I added in debian/rules:

  export DEB_BUILD_PATH=$(CURDIR)

And everything *seemed* to work correctly.

No matter what directory you're in:

  -ffile-prefix-map=/path/to/build/dir=.

So when calling from directory /path/to/build/dir/A/B/C any embedded
paths should get remapped to ./A/B/C, and from directory
/path/to/build/dir/A/B/C/.. should get remapped to ./A/B/C/.. I think.


This is less complicated that the BUILD_PATH_PREFIX_MAP specifications
that Ximin Luo had earlier tried to get accepted into gcc; I do recall
that tried to handle more complicated cases...


live well,
  vagrant
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#985553; Package dpkg. (Tue, 24 Aug 2021 01:39:03 GMT) (full text, mbox, link).


Acknowledgement sent to Guillem Jover <guillem@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Tue, 24 Aug 2021 01:39:03 GMT) (full text, mbox, link).


Message #20 received at 985553@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>
To: Vagrant Cascadian <vagrant@reproducible-builds.org>, 985553@bugs.debian.org
Subject: Re: Bug#985553: dpkg: RFC use gcc .spec files for fixfilepath/fixdebugpath
Date: Tue, 24 Aug 2021 03:37:06 +0200
On Mon, 2021-08-23 at 17:07:13 -0700, Vagrant Cascadian wrote:
> On 2021-08-23, Guillem Jover wrote:
> > On Fri, 2021-03-19 at 11:32:52 -0700, Vagrant Cascadian wrote:
> >> Note that it is only a partial patch; it doesn't export DEB_BUILD_PATH
> >> in dpkg-buildpackage and/or dpkg-buildflags; help as to the best place
> >> to implement that would be appreciated!
> >
> > The ideal place to export such variable would be from
> > dpkg-buildpackage, but sadly as long as that's not the canonical entry
> > point for building packages, I'm seeing only doing that if the
> > implementation can avoid assuming the variable is set and shows no
> > regressions in that case.
> 
> I noticed SOURCE_DATE_EPOCH was defined in:
> 
>   /usr/share/dpkg/pkg-info.mk:SOURCE_DATE_EPOCH ?= $(call dpkg_late_eval,SOURCE_DATE_EPOCH,dpkg-parsechangelog -STimestamp)
> 
> Would it make sense to do the same for DEB_BUILD_PATH? Or would CURDIR
> not be the right value at that time? Are there cases where
> SOURCE_DATE_EPOCH isn't defined when building packages (e.g. fakeroot
> debian/rules build). From a reproducible builds perspective, it seems
> SOURCE_DATE_EPOCH is working well enough, and defining DEB_BUILD_PATH in
> the same place should be good enough too.

The difference being that a missing SOURCE_DATE_EPOCH causes at most
"just" an unreproducible build. While a missing DEB_BUILD_PATH
currently (AFAICS) would break the build. :/ See below.

> >> From 2f2985a12673866a341551399a5461fe8d7e617b Mon Sep 17 00:00:00 2001
> >> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
> >> Date: Sun, 14 Mar 2021 00:19:56 -0800
> >> Subject: [PATCH] Implement reproducible=(fixfilepath|fixdebugpath) as spec
> >>  files.
> >
> >> diff --git a/data/fixdebugpath.specs b/data/fixdebugpath.specs
> >> new file mode 100644
> >> index 000000000..e1cc16b0a
> >> --- /dev/null
> >> +++ b/data/fixdebugpath.specs
> >> @@ -0,0 +1,2 @@
> >> +*self_spec:
> >> ++ -fdebug-prefix-map=%:getenv(DEB_BUILD_PATH =.)
> >
> > I'm not sure the equal sign is correct here?
> 
> I'm not sure what's wrong with it ...
> 
> Currently fixdebugpath would set something like:
> 
>   -fdebug-prefix-map=/absolute/path/to/build/dir=.
> 
> This gets the value of DEB_BUILD_PATH from an environment variable
> instead.
> 
> Or is there something I'm not grasping with how .spec files work?

Ah, misread the function documentation, and I see now that it just
gets appended to the value from the envvar, and is not a fallback
value. Sorry! Although now that I reread this carefully I see that
it's perhaps worse than I thought, as there's this part:

     'getenv'
          The 'getenv' spec function takes two arguments: an environment
          variable name and a string.  If the environment variable is
          not defined, a fatal error is issued.

That would mean that if the environment variable is not defined, we
fail the build. :(

> […] as long as you set the DEB_BUILD_PATH environment variable
> only once per build... it should effectively be a fixed value.

But that's a problem, because unfortunately debian/rules is still a
supported package building entry point, when using that directly there
is no guarantee that DEB_BUILD_PATH would be set.  Which would make the
build fail.

> When I tested this, I added in debian/rules:
> 
>   export DEB_BUILD_PATH=$(CURDIR)
> 
> And everything *seemed* to work correctly.

Sure, but that's the thing we cannot guarantee to be happening. :)

So, one option would be to make dpkg-buildflags try to look for the
debian/ directory to try to determine the source tree root dir.
Unfortunately I don't think that'd be reliable, as I could imagine
there being calls from subdirs containing debian/ hierarchies and
similar, even though that might be contrived, but dpkg-buildflags has
never required being called from within a Debian source tree.

(If only we could finally just make dpkg-buildpackage the only
supported entry point…)

Thanks,
Guillem



Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#985553; Package dpkg. (Tue, 24 Aug 2021 02:27:03 GMT) (full text, mbox, link).


Acknowledgement sent to Vagrant Cascadian <vagrant@reproducible-builds.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Tue, 24 Aug 2021 02:27:03 GMT) (full text, mbox, link).


Message #25 received at 985553@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>
To: Guillem Jover <guillem@debian.org>, 985553@bugs.debian.org
Subject: Re: Bug#985553: dpkg: RFC use gcc .spec files for fixfilepath/fixdebugpath
Date: Mon, 23 Aug 2021 19:22:54 -0700
[Message part 1 (text/plain, inline)]
On 2021-08-24, Guillem Jover wrote:
> On Mon, 2021-08-23 at 17:07:13 -0700, Vagrant Cascadian wrote:
>> On 2021-08-23, Guillem Jover wrote:
>> > On Fri, 2021-03-19 at 11:32:52 -0700, Vagrant Cascadian wrote:
>> >> Note that it is only a partial patch; it doesn't export DEB_BUILD_PATH
>> >> in dpkg-buildpackage and/or dpkg-buildflags; help as to the best place
>> >> to implement that would be appreciated!
>> >
>> > The ideal place to export such variable would be from
>> > dpkg-buildpackage, but sadly as long as that's not the canonical entry
>> > point for building packages, I'm seeing only doing that if the
>> > implementation can avoid assuming the variable is set and shows no
>> > regressions in that case.
>> 
>> I noticed SOURCE_DATE_EPOCH was defined in:
>> 
>>   /usr/share/dpkg/pkg-info.mk:SOURCE_DATE_EPOCH ?= $(call dpkg_late_eval,SOURCE_DATE_EPOCH,dpkg-parsechangelog -STimestamp)
>> 
>> Would it make sense to do the same for DEB_BUILD_PATH? Or would CURDIR
>> not be the right value at that time? Are there cases where
>> SOURCE_DATE_EPOCH isn't defined when building packages (e.g. fakeroot
>> debian/rules build). From a reproducible builds perspective, it seems
>> SOURCE_DATE_EPOCH is working well enough, and defining DEB_BUILD_PATH in
>> the same place should be good enough too.
>
> The difference being that a missing SOURCE_DATE_EPOCH causes at most
> "just" an unreproducible build. While a missing DEB_BUILD_PATH
> currently (AFAICS) would break the build. :/ See below.
...
> Ah, misread the function documentation, and I see now that it just
> gets appended to the value from the envvar, and is not a fallback
> value. Sorry! Although now that I reread this carefully I see that
> it's perhaps worse than I thought, as there's this part:
>
>      'getenv'
>           The 'getenv' spec function takes two arguments: an environment
>           variable name and a string.  If the environment variable is
>           not defined, a fatal error is issued.
>
> That would mean that if the environment variable is not defined, we
> fail the build. :(

Got it. So in order to use this we would need to ensure any used
environment variable is *always* set...

>> […] as long as you set the DEB_BUILD_PATH environment variable
>> only once per build... it should effectively be a fixed value.
>
> But that's a problem, because unfortunately debian/rules is still a
> supported package building entry point, when using that directly there
> is no guarantee that DEB_BUILD_PATH would be set.  Which would make the
> build fail.
>
>> When I tested this, I added in debian/rules:
>> 
>>   export DEB_BUILD_PATH=$(CURDIR)
>> 
>> And everything *seemed* to work correctly.
>
> Sure, but that's the thing we cannot guarantee to be happening. :)
>
> So, one option would be to make dpkg-buildflags try to look for the
> debian/ directory to try to determine the source tree root dir.
> Unfortunately I don't think that'd be reliable, as I could imagine
> there being calls from subdirs containing debian/ hierarchies and
> similar, even though that might be contrived, but dpkg-buildflags has
> never required being called from within a Debian source tree.
>
> (If only we could finally just make dpkg-buildpackage the only
> supported entry point…)

I see.

Ok, a slightly more complicated workflow pseudocode:

if DEB_BUILD_PATH && fixfilepath
then
  -spec=/usr/share/dpkg/fixfilepath.spec ...
else if fixfilepath
  -ffile-prefix-map= build_path ...
fi

In other words, only pass the spec file that requires DEB_BUILD_PATH to
be set when DEB_BUILD_PATH *is* set, falling back to the previous
behavior setting ffile-prefix-map directly?

Another pseudocode idea:

if fixfilepath
then
  setenv DEB_BUILD_PATH $build_path
  export DEB_BUILD_PATH
  -spec=/usr/share/dpkg/fixfilepath.spec
fi


live well,
  vagrant
[signature.asc (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Dpkg Developers <debian-dpkg@lists.debian.org>:
Bug#985553; Package dpkg. (Tue, 24 Aug 2021 09:09:07 GMT) (full text, mbox, link).


Acknowledgement sent to Guillem Jover <guillem@debian.org>:
Extra info received and forwarded to list. Copy sent to Dpkg Developers <debian-dpkg@lists.debian.org>. (Tue, 24 Aug 2021 09:09:07 GMT) (full text, mbox, link).


Message #30 received at 985553@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>
To: Vagrant Cascadian <vagrant@reproducible-builds.org>, 985553@bugs.debian.org
Subject: Re: Bug#985553: dpkg: RFC use gcc .spec files for fixfilepath/fixdebugpath
Date: Tue, 24 Aug 2021 11:07:27 +0200
On Mon, 2021-08-23 at 19:22:54 -0700, Vagrant Cascadian wrote:
> Ok, a slightly more complicated workflow pseudocode:
> 
> if DEB_BUILD_PATH && fixfilepath
> then
>   -spec=/usr/share/dpkg/fixfilepath.spec ...
> else if fixfilepath
>   -ffile-prefix-map= build_path ...
> fi

While in theory something like this might work, I'm afraid this is
still a potentially pretty dangerous thing to do. The problem I see is
that we'd be checking for the presence of an envvar deep in a call
stack that is going to be completely independent from the ones that
are going to be evaluating the spec file with its own envvar fetching,
where something could have set the envvar for the first branch but not
the second (or unset it). But see down below…

> In other words, only pass the spec file that requires DEB_BUILD_PATH to
> be set when DEB_BUILD_PATH *is* set, falling back to the previous
> behavior setting ffile-prefix-map directly?
> 
> Another pseudocode idea:
> 
> if fixfilepath
> then
>   setenv DEB_BUILD_PATH $build_path
>   export DEB_BUILD_PATH
>   -spec=/usr/share/dpkg/fixfilepath.spec
> fi

I don't think that's workable, as per the above.


There's also the perennial problem with GNU make not exposing variables
exported within a Makefile into $(shell) directives, but I guess in
those scenarios that might work in our favor in most cases.

  ,--- test.mk ---
  export DEB_BUILD_PATH = something
  all:
  	echo $(shell dpkg-buildflags | grep something)
  `---

Something I've noticed now is that buildflags.mk is missing passing
DEB_BUILD_PATH explicitly to the dpkg-buildflags call, so I'm merging
the following into git main:

diff --git i/scripts/mk/buildflags.mk w/scripts/mk/buildflags.mk
index 442b7d671..f7ebe8f2c 100644
--- i/scripts/mk/buildflags.mk
+++ w/scripts/mk/buildflags.mk
@@ -31,6 +31,7 @@ endef
 
 $(eval $(call dpkg_buildflags_export_envvar,DEB_BUILD_OPTIONS))
 $(eval $(call dpkg_buildflags_export_envvar,DEB_BUILD_MAINT_OPTIONS))
+$(eval $(call dpkg_buildflags_export_envvar,DEB_BUILD_PATH))
 $(foreach flag,$(DPKG_BUILDFLAGS_LIST),\
   $(foreach operation,SET STRIP APPEND PREPEND,\
     $(eval $(call dpkg_buildflags_export_envvar,DEB_$(flag)_MAINT_$(operation)))))


And after checking with codesearch.d.o, I see only Linux is setting
DEB_BUILD_PATH explicitly from debian/*. But that does not guarantee
build machinery is not resetting envvars f.ex. which would break the
build.

Perhaps a soothing check would be to do an archive rebuild using the
first workflow you proposed above (w/ and w/o setting DEB_BUILD_PATH
externally)? Would that be too cumbersome?

If that's successful, and even though that still has the potential to
break stuff (including non-Debian source packages), at that point it
might be OKish to set it from dpkg-buildpackage because even if it is
not set I'd expect things to be more or less fine.

Thanks,
Guillem



Severity set to 'wishlist' from 'normal' Request was from Guillem Jover <guillem@debian.org> to control@bugs.debian.org. (Sun, 27 Mar 2022 22:57:07 GMT) (full text, mbox, link).


Added tag(s) moreinfo. Request was from Guillem Jover <guillem@debian.org> to control@bugs.debian.org. (Sat, 17 Sep 2022 07:42:03 GMT) (full text, mbox, link).


Bug reassigned from package 'dpkg' to 'dpkg-dev'. Request was from Guillem Jover <guillem@debian.org> to control@bugs.debian.org. (Wed, 12 Oct 2022 17:21:07 GMT) (full text, mbox, link).


Changed Bug title to 'dpkg-buildflags: RFC use gcc .spec files for fixfilepath/fixdebugpath' from 'dpkg: RFC use gcc .spec files for fixfilepath/fixdebugpath'. Request was from Guillem Jover <guillem@debian.org> to control@bugs.debian.org. (Wed, 12 Oct 2022 17:21:07 GMT) (full text, mbox, link).


Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Wed May 17 11:06:00 2023; Machine Name: buxtehude

Debian Bug tracking system

Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.

Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.