Debian Bug report logs - #848129
pie-link specs should not be passed when pie is not enabled

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@debian.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: pie-link specs should not be passed when pie is not enabled

Date: Wed, 14 Dec 2016 12:54:41 +0100

Package: dpkg-dev
Version: 1.18.15
Severity: important
Tags: sid stretch

This is seen on all architectures where pie is not enabled by default. These
specs should not be passed when pie is not in effect.  Seen only when looking at
the python2.7 ftbfs on x32.  And verified that the python2.7 build succeeds
again when the specs are not passed.

Message #10 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: Matthias Klose <doko@debian.org>, 848129@bugs.debian.org

Subject: Re: Bug#848129: pie-link specs should not be passed when pie is not enabled

Date: Thu, 15 Dec 2016 13:27:03 +0100

Hi!

On Wed, 2016-12-14 at 12:54:41 +0100, Matthias Klose wrote:
> Package: dpkg-dev
> Version: 1.18.15
> Severity: important
> Tags: sid stretch
> 
> This is seen on all architectures where pie is not enabled by default. These
> specs should not be passed when pie is not in effect.

I assume you mean enabled by default by gcc.

> Seen only when looking at the python2.7 ftbfs on x32.  And verified that
> the python2.7 build succeeds again when the specs are not passed.

If python2.7 does not build with the PIE spec files on x32 that means
that either there's a portability issue on x32 in python2.7, or that
there's a more fundamental problem with x32 and PIE (f.ex. openssl1.0
also only fails on x32 with PIE specs, see #845193, although openssl
builds fine there).

If the latter, I'd be fine with blacklisting PIE on x32 as "not yet
working", so that it cannot be enabled at all from the dpkg side.
(It also might well be that these packages would fail anyway in case
gcc enabled PIE by default on x32.) Otherwise I guess I'll merge this
and the other report and probably either reassign to gcc or close.


The rationale for ending up enabling this globally in dpkg, was that
enabling this partially in gcc means handling this in packaging is an
inconsistent mess.

We'd have some packages building with PIE in some architectures and
not on others (default hardening build flags option), packages building
with PIE enabled everywhere using gcc default or dpkg PIE enabling specs
(with hardening=+pie), or PIE disabled everywhere using PIE disabling
specs or nothing where gcc does not enable it.

So we'd need specs files in some places no matter what. And the option
of just never using specs files, and not making it possible to disable
PIE for example or enable it on other arches explicily would be
inconsistent and a pain for maintainers.

We've also never enabled hardening flags partially, all hardening
flags are always enabled as long as they work, otherwise they are just
globally blacklisted on certain arches, even if the packages requests
them.

I find the current situation pretty suboptimal TBH. :/

Thanks,
Guillem

Message #15 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@debian.org>

To: Guillem Jover <guillem@debian.org>, 848129@bugs.debian.org, Bálint Réczey <balint@balintreczey.hu>

Subject: Re: Bug#848129: pie-link specs should not be passed when pie is not enabled

Date: Thu, 15 Dec 2016 23:26:11 +0100

On 15.12.2016 13:27, Guillem Jover wrote:
> Hi!
> 
> On Wed, 2016-12-14 at 12:54:41 +0100, Matthias Klose wrote:
>> Package: dpkg-dev
>> Version: 1.18.15
>> Severity: important
>> Tags: sid stretch
>>
>> This is seen on all architectures where pie is not enabled by default. These
>> specs should not be passed when pie is not in effect.
> 
> I assume you mean enabled by default by gcc.

No, this particular build log doesn't show any -fPIE/-pie flags, so it is
neither enabled by dpkg-buildflags nor by gcc.

>> Seen only when looking at the python2.7 ftbfs on x32.  And verified that
>> the python2.7 build succeeds again when the specs are not passed.
> 
> If python2.7 does not build with the PIE spec files on x32 that means
> that either there's a portability issue on x32 in python2.7, or that
> there's a more fundamental problem with x32 and PIE (f.ex. openssl1.0
> also only fails on x32 with PIE specs, see #845193, although openssl
> builds fine there).
> 
> If the latter, I'd be fine with blacklisting PIE on x32 as "not yet
> working", so that it cannot be enabled at all from the dpkg side.
> (It also might well be that these packages would fail anyway in case
> gcc enabled PIE by default on x32.) Otherwise I guess I'll merge this
> and the other report and probably either reassign to gcc or close.
> 
> 
> The rationale for ending up enabling this globally in dpkg, was that
> enabling this partially in gcc means handling this in packaging is an
> inconsistent mess.
> 
> We'd have some packages building with PIE in some architectures and
> not on others (default hardening build flags option), packages building
> with PIE enabled everywhere using gcc default or dpkg PIE enabling specs
> (with hardening=+pie), or PIE disabled everywhere using PIE disabling
> specs or nothing where gcc does not enable it.
> 
> So we'd need specs files in some places no matter what. And the option
> of just never using specs files, and not making it possible to disable
> PIE for example or enable it on other arches explicily would be
> inconsistent and a pain for maintainers.

Well, Balint asked to enable the pie defaults in GCC. I wasn't aware that more
work regarding the specs was needed.  Ideally these changes should be done in
one place, not in two.

> We've also never enabled hardening flags partially, all hardening
> flags are always enabled as long as they work, otherwise they are just
> globally blacklisted on certain arches, even if the packages requests
> them.

The pie flags are the first ones which have a performance impact, and probably
were never tested on our non-linux targets. So yes, I'm careful to apply these
on each target.  Please live with it that defaults may differ.

I don't think that dpkg-buildflags should have any business passing spec files
for cases where these spec files are not needed.  This is monkey-patching GCC
for unrelated or convenience reasons.  This did go wrong as seen with #843791,
#843826, and now with at least the python2.7 x32 build.  For the latter it is
not important that the build can/should be fixed or being worked around, but
that just passing these specs is causing side effects.  The GCC packages don't
divert dpkg-buildflags either, and I see this in the end as a diversion as well.
 And I can't find any docs about what these specs are supposed to do.

For the upload of the GCC 6.3.0 release candidate I'm ignoring these spec files
when they are not needed.  I think this is the least invasive option for the
upcoming stretch release. A 'note' message is printed when these options are
ignored, so this shouldn't be an issue with any -Werror option either.

> I find the current situation pretty suboptimal TBH. :/

Balint and the release team asked to enable this.  I raised concerns that this
would be too late for the release cycle, but my concerns were ignored (sorry,
can't find this email in a bug report, must be on some ML).  I'm fine to undo
the pie enabling for stretch and work on a better solution for buster if anybody
requests this.  There are plenty of bug reports where people are not happy with
the pie defaults in GCC.

Ideally I'd like to see a solution where no spec file changes are required and
the the appropriate changes are integrated in GCC upstream.

Matthias

Message #20 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Bálint Réczey <balint@balintreczey.hu>

To: Matthias Klose <doko@debian.org>

Cc: Guillem Jover <guillem@debian.org>, 848129@bugs.debian.org

Subject: Re: Bug#848129: pie-link specs should not be passed when pie is not enabled

Date: Fri, 16 Dec 2016 10:51:53 +0100

2016-12-15 23:26 GMT+01:00 Matthias Klose <doko@debian.org>:
> On 15.12.2016 13:27, Guillem Jover wrote:
>> Hi!
>>
>> On Wed, 2016-12-14 at 12:54:41 +0100, Matthias Klose wrote:
>>> Package: dpkg-dev
>>> Version: 1.18.15
>>> Severity: important
>>> Tags: sid stretch
>>>
>>> This is seen on all architectures where pie is not enabled by default. These
>>> specs should not be passed when pie is not in effect.
>>
>> I assume you mean enabled by default by gcc.
>
> No, this particular build log doesn't show any -fPIE/-pie flags, so it is
> neither enabled by dpkg-buildflags nor by gcc.
>
>>> Seen only when looking at the python2.7 ftbfs on x32.  And verified that
>>> the python2.7 build succeeds again when the specs are not passed.
>>
>> If python2.7 does not build with the PIE spec files on x32 that means
>> that either there's a portability issue on x32 in python2.7, or that
>> there's a more fundamental problem with x32 and PIE (f.ex. openssl1.0
>> also only fails on x32 with PIE specs, see #845193, although openssl
>> builds fine there).
>>
>> If the latter, I'd be fine with blacklisting PIE on x32 as "not yet
>> working", so that it cannot be enabled at all from the dpkg side.
>> (It also might well be that these packages would fail anyway in case
>> gcc enabled PIE by default on x32.) Otherwise I guess I'll merge this
>> and the other report and probably either reassign to gcc or close.
>>
>>
>> The rationale for ending up enabling this globally in dpkg, was that
>> enabling this partially in gcc means handling this in packaging is an
>> inconsistent mess.
>>
>> We'd have some packages building with PIE in some architectures and
>> not on others (default hardening build flags option), packages building
>> with PIE enabled everywhere using gcc default or dpkg PIE enabling specs
>> (with hardening=+pie), or PIE disabled everywhere using PIE disabling
>> specs or nothing where gcc does not enable it.
>>
>> So we'd need specs files in some places no matter what. And the option
>> of just never using specs files, and not making it possible to disable
>> PIE for example or enable it on other arches explicily would be
>> inconsistent and a pain for maintainers.

Porters could opt-in for PIE and architectures not opted-in are left out.

Please don't enable PIE for architectures not opted-in because those are
the architectures with typically fewer users and less manpower to fix issues.

I would love to see PIE enabled for all arches, but this needs more work
than the available manpower.

>
> Well, Balint asked to enable the pie defaults in GCC. I wasn't aware that more
> work regarding the specs was needed.  Ideally these changes should be done in
> one place, not in two.

Yes, I tested and asked for enabling PIE in GCC because this is the low risk
method. It is used by Ubuntu and GCC parameter parsing makes enabling it
from dpkg basically does not work. Setting it through spec files is fragile and
it is not tested well.
Please simply drop using spec files for both setting and disabling PIE. You
may be able to figure out a good solution with full archive rebuilds for
Stretch+1, but there is no time for that now.

The solution of enabling PIE for a subset of arches through GCC and not
enabling it for the rest of the arches has Release Team's blessing thus please
don't divert from that.

>
>> We've also never enabled hardening flags partially, all hardening
>> flags are always enabled as long as they work, otherwise they are just
>> globally blacklisted on certain arches, even if the packages requests
>> them.
>
> The pie flags are the first ones which have a performance impact, and probably
> were never tested on our non-linux targets. So yes, I'm careful to apply these
> on each target.  Please live with it that defaults may differ.
>
> I don't think that dpkg-buildflags should have any business passing spec files
> for cases where these spec files are not needed.  This is monkey-patching GCC
> for unrelated or convenience reasons.  This did go wrong as seen with #843791,
> #843826, and now with at least the python2.7 x32 build.  For the latter it is
> not important that the build can/should be fixed or being worked around, but
> that just passing these specs is causing side effects.  The GCC packages don't
> divert dpkg-buildflags either, and I see this in the end as a diversion as well.
>  And I can't find any docs about what these specs are supposed to do.
>
> For the upload of the GCC 6.3.0 release candidate I'm ignoring these spec files
> when they are not needed.  I think this is the least invasive option for the
> upcoming stretch release. A 'note' message is printed when these options are
> ignored, so this shouldn't be an issue with any -Werror option either.
>
>> I find the current situation pretty suboptimal TBH. :/
>
> Balint and the release team asked to enable this.  I raised concerns that this
> would be too late for the release cycle, but my concerns were ignored (sorry,
> can't find this email in a bug report, must be on some ML).  I'm fine to undo
> the pie enabling for stretch and work on a better solution for buster if anybody
> requests this.  There are plenty of bug reports where people are not happy with
> the pie defaults in GCC.

The concerns were reasonable, we were late, but not too late. They were
listened to and acknowledged that this change involved risks and needed
work but both the risks and the amount of work involved were acceptable.
IMO the Release Team made the right call here.

Please don't revert the GCC changes, the reversion would delay Stretch
more and frankly Stretch without PIE is not a release I would recommend
to anyone.

If dpkg stops passing spec files we are basically done with the PIE
changes.

Upstreams already need to live with GCC-s with PIE enabled by default
due to Ubuntu's 16.10 thus maintainers following new upstream releases
will be covered, the rest is being helped out.

>
> Ideally I'd like to see a solution where no spec file changes are required and
> the the appropriate changes are integrated in GCC upstream.

Me too.

I asked GCC upstream to make PIE easy to disable but it does not seem
to happen:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77464

Cheers,
Balint

Message #25 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: Matthias Klose <doko@debian.org>, 848129@bugs.debian.org

Cc: Bálint Réczey <balint@balintreczey.hu>

Subject: Re: Bug#848129: pie-link specs should not be passed when pie is not enabled

Date: Wed, 18 Jan 2017 04:19:27 +0100

On Thu, 2016-12-15 at 23:26:11 +0100, Matthias Klose wrote:
> On 15.12.2016 13:27, Guillem Jover wrote:
> > On Wed, 2016-12-14 at 12:54:41 +0100, Matthias Klose wrote:
> >> Package: dpkg-dev
> >> Version: 1.18.15
> >> Severity: important
> >> Tags: sid stretch
> >>
> >> This is seen on all architectures where pie is not enabled by default. These
> >> specs should not be passed when pie is not in effect.
> > 
> > I assume you mean enabled by default by gcc.
> 
> No, this particular build log doesn't show any -fPIE/-pie flags, so it is
> neither enabled by dpkg-buildflags nor by gcc.

I only understood this request after reading the gcc patch. So, you meant
not enabled explicitly by the package or by the builder.

> > So we'd need specs files in some places no matter what. And the option
> > of just never using specs files, and not making it possible to disable
> > PIE for example or enable it on other arches explicily would be
> > inconsistent and a pain for maintainers.
> 
> Well, Balint asked to enable the pie defaults in GCC. I wasn't aware that more
> work regarding the specs was needed.  Ideally these changes should be done in
> one place, not in two.

At this point I think it was a mistake to enable this in GCC, given
the current situation…

> I don't think that dpkg-buildflags should have any business passing spec files
> for cases where these spec files are not needed.  This is monkey-patching GCC
> for unrelated or convenience reasons.

They are needed whenever PIE needs to be enabled or disabled, and gcc
does not do that already.

> This did go wrong as seen with #843791, #843826,

These were just bugs, in the same way #849542 was an implementation
bug.

> and now with at least the python2.7 x32 build.  For the latter it is
> not important that the build can/should be fixed or being worked around, but
> that just passing these specs is causing side effects.  The GCC packages don't
> divert dpkg-buildflags either, and I see this in the end as a diversion as well.

gcc in the end does what the upper layers tell it to do, and in the
end the distribution policy is set by the specific packages or by
dpkg-buildflags if the packages decide to use it. By that measure all
packages are diverting gcc whenever they diverge from gcc defaults…

> And I can't find any docs about what these specs are supposed to do.

I'm not sure what kind of documentation you are looking for.

> For the upload of the GCC 6.3.0 release candidate I'm ignoring these spec files
> when they are not needed.  I think this is the least invasive option for the
> upcoming stretch release. A 'note' message is printed when these options are
> ignored, so this shouldn't be an issue with any -Werror option either.

Oh, wow, I saw that, and lost all motivation to discuss this any
further. But there's too much fallout due to this, so it needs to be
addressed one way or another.

That patch is just broken-by-design. It is a complete layer violation. It
also breaks unrelated software such as cmake (#851720) due to that 'note'.
And breaks any package that does not export DEB_BUILD_MAINT_OPTIONS from
debian/rules, such as any using the dpkg architecture.mk fragment or when
calling dpkg-buildflags with the --export=configure or --export=cmdline
options, or similar constructs.

I'll start a thread on d-d, as switching to the behavior that you
request, which I consider pretty suboptimal, impacts porters and
package maintainers. If there's consensus to do the switch then I'll
do that.

> > I find the current situation pretty suboptimal TBH. :/
> 
> Balint and the release team asked to enable this.  I raised concerns that this
> would be too late for the release cycle, but my concerns were ignored (sorry,
> can't find this email in a bug report, must be on some ML).  I'm fine to undo
> the pie enabling for stretch and work on a better solution for buster if anybody
> requests this.  There are plenty of bug reports where people are not happy with
> the pie defaults in GCC.
> 
> Ideally I'd like to see a solution where no spec file changes are required and
> the the appropriate changes are integrated in GCC upstream.

Well, at this point I'd rather remove all traces of PIE support from
dpkg-buildflags and let you and Bálint handle all this mess, unfortunately
this is not possible, as one of the *the* interface to enable this in
packages is precisely dpkg-buildflags…

Regards,
Guillem

Message #30 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: balint@balintreczey.hu, 848129@bugs.debian.org

Cc: Matthias Klose <doko@debian.org>

Subject: Re: Bug#848129: pie-link specs should not be passed when pie is not enabled

Date: Wed, 18 Jan 2017 04:28:12 +0100

On Fri, 2016-12-16 at 10:51:53 +0100, Bálint Réczey wrote:
> 2016-12-15 23:26 GMT+01:00 Matthias Klose <doko@debian.org>:
> > Well, Balint asked to enable the pie defaults in GCC. I wasn't aware that more
> > work regarding the specs was needed.  Ideally these changes should be done in
> > one place, not in two.
> 
> Yes, I tested and asked for enabling PIE in GCC because this is the low risk
> method. It is used by Ubuntu and GCC parameter parsing makes enabling it
> from dpkg basically does not work. Setting it through spec files is fragile and
> it is not tested well.

The -specs files should be pretty robust now. Anything not building
with those would IMO be broken with PIE by default from gcc anyway. And
In the end gcc also drives its frontends via specs files internally.

> Please simply drop using spec files for both setting and disabling PIE.

Sorry but dropping the -specs files completely, even if honoring
Matthias request, would be unnacceptable, as I've mentioned before
multiple times now. Maintainers should be able to disable PIE, not
doing so would leave them with the job of figuring out how to do that,
which might mean having to extensively patch the build system or
similar. Making maintainers work harder just because, is not acceptable.
At that point I'd rather we disable PIE globally, really.

Regards,
Guillem

Message #35 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Drew Parsons <dparsons@debian.org>

To: 848129@bugs.debian.org

Subject: Re: Bug#848129: pie-link specs should not be passed when pie is not enabled

Date: Wed, 22 Feb 2017 18:12:51 +0800

I suspect (or at least wonder if) the FTBFS on petsc [1] might be
relevant to this discussion. I haven't had time to debug fully, but
petsc started failing to build on Tier 2 architectures after the pie
defaults changed. petsc builds libraries, so the build explicitly uses
-fPIC. Currently of the 2nd tiers arches, only hppa builds successfully
(alpha also, until the most recent build).

The failure occurs at configure time when the build checks the shared
library linker.

[1] https://buildd.debian.org/status/package.php?p=petsc&suite=unstable

Message #38 received at 848129-submitter@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: 848129-submitter@bugs.debian.org

Subject: Bug#848129 in package dpkg marked as pending

Date: Sun, 26 Feb 2017 22:50:13 +0000

Control: tag 848129 pending

Hi!

Bug #848129 in package dpkg reported by you has been fixed in
the dpkg/dpkg.git Git repository. You can see the changelog below, and
you can check the diff of the fix at:

    https://anonscm.debian.org/cgit/dpkg/dpkg.git/diff/?id=ce97c58

---
commit ce97c5865788e0d311645d12d1c84f6fdf1412ea
Author: Guillem Jover <guillem@debian.org>
Date:   Tue Feb 7 15:47:23 2017 +0100

    Dpkg::Vendor::Debian: Switch PIE handling to have no default (!)
    
    Delegate the setting to gcc builtin or an explicit request by a user.
    This is needed to cope with the general PIE brokenness situation in
    Debian, and the current specific brokenness of a Debian gcc patch
    mangling the dpkg build flags.
    
    This is wrong in so many levels, as we'll have discrepancies between
    architectures, the interface towards maintainers is inconsistent, and
    updating the PIE support needs touching and coordinating two places. But
    it's certainly the current lesser evil.
    
    Closes: #848129, #845550

diff --git a/debian/changelog b/debian/changelog
index ec8551d..3c98ade 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -17,6 +17,11 @@ dpkg (1.18.23) UNRELEASED; urgency=medium
       Thanks to Nicolas Boulenguez <nicolas@debian.org>.
     - Mark kfreebsd-amd64, kfreebsd-i386, sparc and sparc64 architectures as
       having gcc builtin PIE in Dpkg::Vendor::Debian.
+    - Switch PIE handling in Dpkg::Vendor::Debian to have no default (!) and
+      delegate the setting to gcc or an explicit request by a user. This is
+      needed to cope with the general PIE brokenness situation in Debian, and
+      the current specific brokenness of a Debian gcc patch mangling the dpkg
+      build flags. Closes: #848129, #845550
   * Documentation:
     - Clarify the requirements for deb-conffile(5) pathnames. Closes: #854417
       Proposed by Dieter Adriaenssens <dieter.adriaenssens@gmail.com>.

Message #45 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Thorsten Glaser <tg@mirbsd.de>

To: Guillem Jover <guillem@debian.org>, 845550@bugs.debian.org, 848129@bugs.debian.org, doko@debian.org

Subject: Re: Bug#845550: in package dpkg marked as pending

Date: Mon, 27 Feb 2017 22:45:24 +0000 (UTC)

Guillem Jover dixit:

>    This is wrong in so many levels, as we'll have discrepancies between
>    architectures, the interface towards maintainers is inconsistent, and
>    updating the PIE support needs touching and coordinating two places. But

Not quite: it *only* needs changing in GCC now that dpkg keeps its
hands off the default PIE setting.

>    it's certainly the current lesser evil.

That, yes. It additionally is the better way to do this because the
-specs= stuff was inherently broken (consider compiler wrappers for
other libcs which do their own specs stuff) and otherwise fragile,
while the “PIE is enabled by default in GCC” mechanism is used by
all architectures now and better understood (and having some arch
not defaulting to PIE is a safe (even if less “secure”) failure mode.

Thank you for this change — now I can probably remove all the
special CFLAGS handling from my packages again…

… will this land in stretch? (If not, I’d better keep it in.)

bye,
//mirabilos
-- 
22:20⎜<asarch> The crazy that persists in his craziness becomes a master
22:21⎜<asarch> And the distance between the craziness and geniality is
only measured by the success 18:35⎜<asarch> "Psychotics are consistently
inconsistent. The essence of sanity is to be inconsistently inconsistent

Message #50 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Matthias Klose <doko@debian.org>

To: Thorsten Glaser <tg@mirbsd.de>, Guillem Jover <guillem@debian.org>, 845550@bugs.debian.org, 848129@bugs.debian.org, doko@debian.org

Subject: Re: Bug#845550: in package dpkg marked as pending

Date: Tue, 28 Feb 2017 08:07:14 +0100

On 27.02.2017 23:45, Thorsten Glaser wrote:
> Guillem Jover dixit:
> 
>>    This is wrong in so many levels, as we'll have discrepancies between
>>    architectures, the interface towards maintainers is inconsistent, and
>>    updating the PIE support needs touching and coordinating two places. But
> 
> Not quite: it *only* needs changing in GCC now that dpkg keeps its
> hands off the default PIE setting.
> 
>>    it's certainly the current lesser evil.
> 
> That, yes. It additionally is the better way to do this because the
> -specs= stuff was inherently broken (consider compiler wrappers for
> other libcs which do their own specs stuff) and otherwise fragile,
> while the “PIE is enabled by default in GCC” mechanism is used by
> all architectures now and better understood (and having some arch
> not defaulting to PIE is a safe (even if less “secure”) failure mode.
> 
> Thank you for this change — now I can probably remove all the
> special CFLAGS handling from my packages again…
> 
> … will this land in stretch? (If not, I’d better keep it in.)

yes, same question here (although I'd like to wait for the the current gcc-6 to
migrate first).

Thanks, Matthias

Message #55 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Thorsten Glaser <tg@mirbsd.de>

To: Guillem Jover <guillem@debian.org>, 845550@bugs.debian.org, 848129@bugs.debian.org, doko@debian.org

Subject: ⚠ not enough ⚠ Re: Bug#845550: in package dpkg marked as pending

Date: Wed, 1 Mar 2017 13:20:19 +0000 (UTC)

Dixi quod…

>Thank you for this change — now I can probably remove all the
>special CFLAGS handling from my packages again…

⚠ ⚠ ⚠

Unfortunately, this is NOT enough!

If some package declares hardening=+all then dpkg will STILL
inject the -specs= stuff into various flags, breaking e.g.
gpgme1.0 (or anything else using Qt) in the process (because
it must be built with PIC, not PIE). Using hardening=+all,-pie
works around this but ⓐ needs maintainer interferences and ⓑ
is another architecture inconsistency.

Guillem, _please_, the only way out of this mess is to never
inject the -specs=* stuff at all, or (probably opening up a
new mess) to always inject it on all architectures (then you’d
see just how broken it is).

Sometimes, it’s better to use a more pragmatic solution instead
of a more “correct” one. (Also, why is this even affecting x32…
I thought Doko enabled PIE on all architectures now?)

Thanks,
//mirabilos
-- 
<diogenese> Beware of ritual lest you forget the meaning behind it.
<igli> yeah but it means if you really care about something, don't
    ritualise it, or you will lose it. don't fetishise it, don't
    obsess. or you'll forget why you love it in the first place.

Message #60 received at 848129@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: Thorsten Glaser <tg@mirbsd.de>, 845550@bugs.debian.org

Cc: 848129@bugs.debian.org, doko@debian.org

Subject: Re: Bug#845550: in package dpkg marked as pending

Date: Wed, 1 Mar 2017 17:11:21 +0100

Hi!

On Mon, 2017-02-27 at 22:45:24 +0000, Thorsten Glaser wrote:
> Guillem Jover dixit:
> >    This is wrong in so many levels, as we'll have discrepancies between
> >    architectures, the interface towards maintainers is inconsistent, and
> >    updating the PIE support needs touching and coordinating two places. But
> 
> Not quite: it *only* needs changing in GCC now that dpkg keeps its
> hands off the default PIE setting.

Nope, because as long as gcc still uses a very restricted whitelist
of arches where to enable PIE by default, dpkg still needs to adapt to
pass or not the PIE flags, on those arches. :(

> >    it's certainly the current lesser evil.
> 
> That, yes. It additionally is the better way to do this because the
> -specs= stuff was inherently broken (consider compiler wrappers for
> other libcs which do their own specs stuff) and otherwise fragile,
> while the “PIE is enabled by default in GCC” mechanism is used by
> all architectures now and better understood (and having some arch
> not defaulting to PIE is a safe (even if less “secure”) failure mode.

I'm not sure I follow. The specs stuff is libc independent, gcc uses
internally builtin specs to drive the frontends. specs files are also
necessary to be able to disable PIE even on packages that build both
programs and libraries, otherwise we get FTBFS (and it's the reason
the specs files got introduced in the first place). Also, as stated
above, nope PIE is still not enabled globally in gcc, and worse the
very broken gcc patch overriding the dpkg build flags might or might
not disable these depending on how the variables are exported…

On Wed, 2017-03-01 at 13:20:19 +0000, Thorsten Glaser wrote:
> Dixi quod…
> >Thank you for this change — now I can probably remove all the
> >special CFLAGS handling from my packages again…
> 
> Unfortunately, this is NOT enough!
> 
> If some package declares hardening=+all then dpkg will STILL
> inject the -specs= stuff into various flags, breaking e.g.
> gpgme1.0 (or anything else using Qt) in the process (because
> it must be built with PIC, not PIE). Using hardening=+all,-pie
> works around this but ⓐ needs maintainer interferences and ⓑ
> is another architecture inconsistency.

If we were not using specs files, and passing straight -fPIE -pie,
or even -fno-PIE -no-pie the fallout would be greater, as the spec
files try to make sure not to use PIE when PIC, static or shared
linking has been specified. So maintainers would need to do way
more manual intervention then.

Why do you state that the specs file are completely broken? How
many arches are affected? After the initial fallout, I've only seen
x32 being broken here, and as I stated on the openssl bug, it seems
the cause is generally somewhere else.

> Guillem, _please_, the only way out of this mess is to never
> inject the -specs=* stuff at all,

See above. Replacing specs with straight flags would be way worse,
and would regress release arches, which I'm not planning on doing at
this point in the release cycle. Removing them completely means we
cannot enable/disable PIE at all, which would seem obvious is
unacceptable?

> or (probably opening up a
> new mess) to always inject it on all architectures (then you’d
> see just how broken it is).

As stated above, I'm not going to introduce changes at this point in
the release cycle which affects release architectures. The biggest
issue here is still the broken handling of PIE within gcc, by using
a whitelist…

> Sometimes, it’s better to use a more pragmatic solution instead
> of a more “correct” one.

Here the specs files is actually the pragmatic solution! Using
straight flags breaks stuff. The correct solution would be to actually
just enable PIE globally in gcc and make dpkg never pass specs nor
flags to enable them.

> (Also, why is this even affecting x32…
> I thought Doko enabled PIE on all architectures now?)

Both because PIE is *not* enabled globally, and because x32 seems to
have some issues IMO, either in the toolchain or in specific packages,
etc.

If you want your arch to get out of this mess, file a bug against gcc
to get it to enable PIE by default. It still sucks, because it has to
be requested per-arch, and then coordinated with dpkg, but at this
point, that's really the best you can do. :(

Thanks,
Guillem

Message #69 received at 848129-close@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: 848129-close@bugs.debian.org

Subject: Bug#848129: fixed in dpkg 1.18.23

Date: Mon, 06 Mar 2017 06:18:56 +0000

Source: dpkg
Source-Version: 1.18.23

We believe that the bug you reported is fixed in the latest version of
dpkg, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 848129@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Guillem Jover <guillem@debian.org> (supplier of updated dpkg package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Mon, 06 Mar 2017 05:41:11 +0100
Source: dpkg
Binary: dpkg libdpkg-dev dpkg-dev libdpkg-perl dselect
Architecture: source
Version: 1.18.23
Distribution: unstable
Urgency: medium
Maintainer: Dpkg Developers <debian-dpkg@lists.debian.org>
Changed-By: Guillem Jover <guillem@debian.org>
Description:
 dpkg       - Debian package management system
 dpkg-dev   - Debian package development tools
 dselect    - Debian package management front-end
 libdpkg-dev - Debian package management static library
 libdpkg-perl - Dpkg perl modules
Closes: 845550 848129 849944 854417 854536 855450 856325 856326
Changes:
 dpkg (1.18.23) unstable; urgency=medium
 .
   * Handle unmatched arch-qualified virtual packages in dpkg-genbuildinfo,
     instead of letting perl die. Closes: #849944
   * Declare .buildinfo format as stable with version 1.0.
   * Do not depend on cxxabi.h to have declared __cxa_pure_virtual, use
     the same “__cxxabiv1” namespace as specified in the C++ ABI, instead
     of using the “abi” alias intended for use by userland.
     Thanks to Jörg Sonnenberger <joerg@netbsd.org>.
   * Add a comment on any C code switch case that falls through. Fixes new
     gcc-7 warnings.
   * Use snprintf() instead of sprintf() in libdpkg when constructing the ar
     member header, as we might overflow depending on the input data.
   * Portability:
     - Do not redeclare sys_siglist in libcompat when the system does so.
       Thanks to Thomas Klausner <wiz@NetBSD.org>.
     - Rename err variable to ret in start-stop-daemon as the former is a
       function on BSDs.
     - Use 5-argument kvm_getprocs() call form on OpenBSD in start-stop-daemon.
     - Use correct struct kinfo_proc ruid submember name on NetBSD in
       start-stop-daemon.
     - Define _KMEMUSER for NetBSD to get declarations for various
       struct kinfo_proc members in start-stop-daemon.
   * Perl modules:
     - Do not special case EM_SPARC32PLUS for NetBSD in Dpkg::Shlibs::Objdump,
       the code has been fixed in NetBSD as that situation could not happen.
     - Fix read() error handling in Dpkg::Shlibs::Objdump::get_format() to
       gracefully ignore non-ELF files again. Closes: #854536
     - Emit an explicit warning from Dpkg::Shlibs::Objdump::Object::analyze()
       for unknown executable formats instead of relying on objdump doing so.
     - Do not parse bogus ELF binaries in Dpkg::Shlibs::Objdump::get_format().
       Reported by Niels Thykier <niels@thykier.net>.
     - Add ‘.mnt-ignore’ to the default ignore lists in Dpkg::Source::Package,
       as we were already ignoring the ‘_MTN’ pathnames. Closes: #855450
       Thanks to Nicolas Boulenguez <nicolas@debian.org>.
     - Mark kfreebsd-amd64, kfreebsd-i386, sparc and sparc64 architectures as
       having gcc builtin PIE in Dpkg::Vendor::Debian.
     - Switch PIE handling in Dpkg::Vendor::Debian to have no default (!) and
       delegate the setting to gcc or an explicit request by a user. This is
       needed to cope with the general PIE brokenness situation in Debian, and
       the current specific brokenness of a Debian gcc patch mangling the dpkg
       build flags. Closes: #848129, #845550
   * Documentation:
     - Clarify the requirements for deb-conffile(5) pathnames. Closes: #854417
       Proposed by Dieter Adriaenssens <dieter.adriaenssens@gmail.com>.
     - Document dpkg-source --before-build and --after-build in --help output.
     - Document dpkg-buildpackage --ignore-builtin-builddeps in --help output.
   * Build system:
     - Check <sys/proc.h> by also including <sys/param.h>, on several BSD
       systems the header is not self-contained.
     - Handle libmd implementations built into system libc, as found on some
       BSD systems.
     - Do not fail on missing compression libraries or headers on automatic
       detection mode. Regression introduced in dpkg 1.18.14.
   * Test suite:
     - Use the detected perl interpreter instead of a random one from PATH.
 .
   [ Updated programs translations ]
   * Dutch (Frans Spiesschaert). Closes: #856325
 .
   [ Updated scripts translations ]
   * German (Helge Kreutzmann).
 .
   [ Updated man pages translations ]
   * Dutch (Frans Spiesschaer). Closes: #856326
Checksums-Sha1:
 687fc2cc5d75609d3f9c64dd6d1271a274e8824c 2032 dpkg_1.18.23.dsc
 a090c0003d27bd467b9d4e683f2fa634f88d9486 4516252 dpkg_1.18.23.tar.xz
 bea5fcdd1181b693cc1d87776a56b61d04ce02b6 7344 dpkg_1.18.23_amd64.buildinfo
Checksums-Sha256:
 5e2fe064fb36d95d6459d2e117df9b54618df5e58925db117b5dd0c74cba2ec4 2032 dpkg_1.18.23.dsc
 cc08802a0cea2ccd0c10716bc71531ff9b9234dd454b83a59f71117a37f36923 4516252 dpkg_1.18.23.tar.xz
 267c599bef3a28f84247a438449a633a2b24200f3e506586a734f18c17375270 7344 dpkg_1.18.23_amd64.buildinfo
Files:
 7d31b1e07896d2464f4c970d2e839010 2032 admin required dpkg_1.18.23.dsc
 2195338c1792b0678575309a099d2da8 4516252 admin required dpkg_1.18.23.tar.xz
 c5479e155ee1e00b98f815e38dfee410 7344 admin required dpkg_1.18.23_amd64.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEETz509DYFDBD1aWV0uXK/PqSuV6MFAli8+MEACgkQuXK/PqSu
V6PJjxAAnQXT2CNiFg1U0Lizr+PfwlT24BMi1Za/J4zAIIjNe3CrP1z1dZpW33Ci
hG5HbacX9Ny7W0MoiaX91jg7oLel0ofYelGflXEO9ixod6EhqqMwb4WR0a0UIArj
XX6sjhVE2L8cOjceODbKF+xerAv5kQW0kjGxgVXhQlvOcl7Ncdpgw0Us9lswVM4m
RPsUm3V9imcJdMaxbYdsZHHs61NESLVpMWnndZzcbc5GWss8Gl6yAkY/cqfYqCi7
goFU8nxpyaKdmbGxM6+T0RBujuJI3oLR4S1COMs6CHCCtGmO6LIWSyAd5GbODd+5
HNrutqqTgKMhtWVIj69MfZzMBOSWQOmB8FiaXAUrEyGE4bOg8PDsGtiOtf4TBgtn
0x6pSRRLLbDiljmw108BSevLZ0R87E5fmtIX7ybFmdb18HVaBv5L7ETRNy1tgjXy
zPS15lun/yDHbRc30WJ862fS6k3UA/ZpGg+qoxWreqMIp13tSqfgDFkki/1fAjkJ
Zti0lG41pR6wCX/VaL3Sa9ZnKMsmqZoUngu1DAXFXqVQGnmRZRKAtegPGEGHsK+J
Xqpz4r/TleUMmB8JfN50Aify7glKgqphjWLoM5CVgOD7SaLD3SOBH9ePRUz51KNr
bHKVwJTtjP0jrM7WOGYIw6sjxdpTJQiUWBoOmgletcygmI092lE=
=nkIW
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.