Debian Bug report logs -
#792687
gettext: please support timestamps from environment
Reported by: Dhole <dhole@openmailbox.org>
Date: Fri, 17 Jul 2015 14:12:01 UTC
Severity: wishlist
Tags: patch, upstream
Merged with 825436
Found in versions gettext/0.19.4-1, gettext/0.19.7-2
Fixed in version gettext/0.19.8.1-3
Done: Santiago Vila <sanvila@debian.org>
Bug is archived. No further changes may be made.
Toggle useless messages
Report forwarded
to debian-bugs-dist@lists.debian.org, reproducible-builds@lists.alioth.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Fri, 17 Jul 2015 14:12:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Dhole <dhole@openmailbox.org>:
New Bug report received and forwarded. Copy sent to reproducible-builds@lists.alioth.debian.org, Santiago Vila <sanvila@debian.org>.
(Fri, 17 Jul 2015 14:12:05 GMT) (full text, mbox, link).
Message #5 received at submit@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
Source: gettext
Version: 0.19.4-1
Severity: wishlist
Tags: patch
User: reproducible-builds@lists.alioth.debian.org
Usertags: toolchain timestamps
X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org
Hi,
While working on the "reproducible builds" effort [1], we have noticed
that the xgettext tool from gettext embeds timestamps on the creation of
PO files.
For the Reproducible Builds effort we are proposing an environment
variable (SOURCE_DATE_EPOCH) [2] that will contain a deterministic epoch
timestamp (based on the latest debian/changelog entry) that could be
used, which should be automatically exported by debhelper in the future [3].
The attached patch proposes a way to use this variable to get
reproducible timestamps in the PO files generated by xgettext, if the
variable has been set (if not, it falls back to the old behavior).
With the attached patch packages using xgettext would then automatically
generate reproducible translation files.
[1]: https://wiki.debian.org/ReproducibleBuilds
[2]: https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal
[3]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=791815
Regards,
--
Dhole
[gettext.diff.txt (text/plain, attachment)]
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Wed, 05 Aug 2015 18:18:06 GMT) (full text, mbox, link).
Acknowledgement sent
to Dhole <dhole@openmailbox.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Wed, 05 Aug 2015 18:18:06 GMT) (full text, mbox, link).
Message #10 received at 792687@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On 07/17/2015 04:08 PM, Dhole wrote:
> Source: gettext
> Version: 0.19.4-1
> Severity: wishlist
> Tags: patch
> User: reproducible-builds@lists.alioth.debian.org
> Usertags: toolchain timestamps
> X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org
>
> Hi,
>
> While working on the "reproducible builds" effort [1], we have noticed
> that the xgettext tool from gettext embeds timestamps on the creation of
> PO files.
>
> For the Reproducible Builds effort we are proposing an environment
> variable (SOURCE_DATE_EPOCH) [2] that will contain a deterministic epoch
> timestamp (based on the latest debian/changelog entry) that could be
> used, which should be automatically exported by debhelper in the future [3].
>
> The attached patch proposes a way to use this variable to get
> reproducible timestamps in the PO files generated by xgettext, if the
> variable has been set (if not, it falls back to the old behavior).
> With the attached patch packages using xgettext would then automatically
> generate reproducible translation files.
>
>
> [1]: https://wiki.debian.org/ReproducibleBuilds
> [2]: https://wiki.debian.org/ReproducibleBuilds/TimestampsProposal
> [3]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=791815
>
> Regards,
>
>
>
> _______________________________________________
> Reproducible-builds mailing list
> Reproducible-builds@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
>
Hi
I'm attaching an update of the submitted patch targeting the latest
gettext version from Debian: 0.19.5.1-1
Regards,
--
Dhole
[gettext.diff.txt (text/plain, attachment)]
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to debian-bugs-dist@lists.debian.org:
Bug#792687; Package src:gettext.
(Mon, 31 Aug 2015 19:36:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Santiago Vila <sanvila@debian.org>:
Extra info received and forwarded to list.
(Mon, 31 Aug 2015 19:36:03 GMT) (full text, mbox, link).
Message #15 received at 792687@bugs.debian.org (full text, mbox, reply):
On Fri, 17 Jul 2015, Dhole wrote:
> Source: gettext
> Version: 0.19.4-1
> Severity: wishlist
> Tags: patch
> User: reproducible-builds@lists.alioth.debian.org
> Usertags: toolchain timestamps
> X-Debbugs-Cc: reproducible-builds@lists.alioth.debian.org
>
> Hi,
>
> While working on the "reproducible builds" effort [1], we have noticed
> that the xgettext tool from gettext embeds timestamps on the creation of
> PO files.
Ok, after looking at many of the packages in reproducible.debian.net
having the tag "lc_messages_randomness" I have to say that your bug
report is not really an accurate description of the problem.
In the gettext system, there are PO files, containing translations:
msgid "hello"
msgstr "hola"
and POT files which are just templates:
msgid "hello"
msgstr ""
serving as the starting point for a new translation in a new language.
Those files (PO and POT) are different and should not be mixed
in a big bag called "PO files" including both.
The xgettext tool is used to generate a POT file containing all
translatable messages from the source code in which those messages
actually appear.
The problem here is that some packages execute xgettext *every* single
time the package is built (creating a different .pot file each time),
and then they also perform msgmerge with the new .pot file and the
existing .po files, creating different .po files each time.
None of that is necessary at all:
For a given source package and debian revision, you only need to
create the .pot file once, for example, just before dpkg-buildpackage.
The same is true for the .po files: If they need to be refreshed,
updated, or whatever, this could and probably should be done
before dpkg-buildpackage as well.
As a proof of concept, I've fixed "pppconfig" the way I think this
kind of bugs should be fixed: By not running "po/update.sh" each and
every time.
Now, my problem is that there is a forked gettext ~reproducible1 and
every build in reproducible.debian.net uses the forked version.
This way the real problem (the fact that a package modifies its own
source at every build) is hidden and more difficult to fix.
Would be possible to drop the forked gettext and try to fix this in
the affected packages?
Thanks.
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Mon, 31 Aug 2015 22:03:07 GMT) (full text, mbox, link).
Acknowledgement sent
to Chris Lamb <lamby@debian.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Mon, 31 Aug 2015 22:03:07 GMT) (full text, mbox, link).
Message #20 received at 792687@bugs.debian.org (full text, mbox, reply):
> As a proof of concept, I've fixed "pppconfig" the way I think this
> kind of bugs should be fixed: By not running "po/update.sh" each and
> every time.
https://gist.github.com/be5b243cfcaaa27819f0 is the diff for anyone
interested
> Would be possible to drop the forked gettext and try to fix this in
> the affected packages?
Seems reasonable, especially as it's only ~40 packages. Would you guess
most of them are fixable under debian/ or require upstream patches?
Regards,
--
,''`.
: :' : Chris Lamb
`. `'` lamby@debian.org / chris-lamb.co.uk
`-
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Tue, 01 Sep 2015 10:12:04 GMT) (full text, mbox, link).
Acknowledgement sent
to Dhole <dhole@openmailbox.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Tue, 01 Sep 2015 10:12:04 GMT) (full text, mbox, link).
Message #25 received at 792687@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On 09/01/2015 12:36 AM, Santiago Vila wrote:
> On Tue, Sep 01, 2015 at 12:01:45AM +0200, Chris Lamb wrote:
>>> As a proof of concept, I've fixed "pppconfig" the way I think this
>>> kind of bugs should be fixed: By not running "po/update.sh" each and
>>> every time.
>>
>> https://gist.github.com/be5b243cfcaaa27819f0 is the diff for anyone
>> interested
>
> Well, that includes some cleanup I also did because the package was
> orphaned (like removing "source and diff are obsolete" warning message
> which is 15 years old at least).
>
> The size of "not running po/update.sh" is really a single line removal.
>
>>> Would be possible to drop the forked gettext and try to fix this in
>>> the affected packages?
>>
>> Seems reasonable, especially as it's only ~40 packages. Would you guess
>> most of them are fixable under debian/ or require upstream patches?
>
> I would bet that most of them are fixable by changing debian/* only
> (but I really don't know).
>
As I understand it, the changes in debian/* would be to disable the call
to po/update.sh. But then there's also changes needed inside po/ because
the maintainer would need to run update.sh (which would modify files in
po/) before packaging right? I don't find it clear if the update.sh
should be run by the Debian maintainer or by upstream.
> It's funny that the current gettext fork in alioth did not seem to
> help about 8 of them:
>
> https://reproducible.debian.net/issues/unstable/lc_messages_randomness_issue.html
>
> (There are 12 unreproducible, but only 8 suffer currently from this issue)
>
The reason why 8 of them are not reproducible even with the patched
gettext is because the calls to xgettext in those packages don't happen
under dh, which is one of the current patched toolchains that export
SOURCE_DATE_EPOCH. Since that variable is not exported, xgettext embeds
the current date. But that happens to other packages as well with other
issues (where the toolchain has already been patched to honour
SOURCE_DATE_EPOCH).
--
Dhole
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to debian-bugs-dist@lists.debian.org:
Bug#792687; Package src:gettext.
(Tue, 01 Sep 2015 11:18:04 GMT) (full text, mbox, link).
Acknowledgement sent
to Santiago Vila <sanvila@debian.org>:
Extra info received and forwarded to list.
(Tue, 01 Sep 2015 11:18:04 GMT) (full text, mbox, link).
Message #30 received at 792687@bugs.debian.org (full text, mbox, reply):
On Tue, Sep 01, 2015 at 12:09:06PM +0200, Dhole wrote:
> As I understand it, the changes in debian/* would be to disable the call
> to po/update.sh. But then there's also changes needed inside po/ because
> the maintainer would need to run update.sh (which would modify files in
> po/) before packaging right? I don't find it clear if the update.sh
> should be run by the Debian maintainer or by upstream.
I consider updating po files to be an "upstream" task.
Sometimes upstream and Debian maintainer are the same person (think
about debconf translations or debian native packages), but even in
such case, updating the files does only need to be done before
dpkg-buildpackage, i.e. only once.
Our task as Debian maintainers is to prepare a source package which is
to be built by the different buildds. It is a little bit odd that
every buildd has to msgmerge the .po files with the updated .pot file.
That's a task for whoever prepares the source package, be it the
Debian maintainer for debconf translations and the like, or the
upstream author for the translations that belong to the upstream
package itself.
> > It's funny that the current gettext fork in alioth did not seem to
> > help about 8 of them:
> >
> > https://reproducible.debian.net/issues/unstable/lc_messages_randomness_issue.html
> >
> > (There are 12 unreproducible, but only 8 suffer currently from this issue)
> >
>
> The reason why 8 of them are not reproducible even with the patched
> gettext is because the calls to xgettext in those packages don't happen
> under dh, which is one of the current patched toolchains that export
> SOURCE_DATE_EPOCH. Since that variable is not exported, xgettext embeds
> the current date. But that happens to other packages as well with other
> issues (where the toolchain has already been patched to honour
> SOURCE_DATE_EPOCH).
Ok, thanks for the clarification.
So, before telling maintainers "switch to dh and all your LC_MESSAGES
unreproducibility problems will be solved" I'd like they to be aware
that updating po files only once (before dpkg-buildpackage) would be
enough for this to be fixed.
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Wed, 09 Sep 2015 00:21:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Reproducible Builds discussion list <reproducible-builds@lists.alioth.debian.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Wed, 09 Sep 2015 00:21:03 GMT) (full text, mbox, link).
Message #35 received at 792687@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On 08/31/2015 09:32 PM, Santiago Vila wrote:
> Would be possible to drop the forked gettext and try to fix this in
> the affected packages?
Hi Santiago, I saw that you have been working on the packages affected
by the POT creation date timestamps in gettext mo files today so maybe
you know this already, but just to be sure:
The patched gettext has been dropped from our reproducible APT
repository, and all the affected packages by the issue
different_pot_creation_date_in_gettext_mo_files (formerly
lc_messages_randomness) have been rebuilt in our jenkins setup, using
the gettext version from Debian unstable:
https://reproducible.debian.net/issues/unstable/different_pot_creation_date_in_gettext_mo_files_issue.html
Regards,
--
Dhole
[signature.asc (application/pgp-signature, attachment)]
Information forwarded
to debian-bugs-dist@lists.debian.org:
Bug#792687; Package src:gettext.
(Wed, 09 Sep 2015 14:12:10 GMT) (full text, mbox, link).
Acknowledgement sent
to Santiago Vila <sanvila@debian.org>:
Extra info received and forwarded to list.
(Wed, 09 Sep 2015 14:12:10 GMT) (full text, mbox, link).
Message #40 received at 792687@bugs.debian.org (full text, mbox, reply):
On Wed, Sep 09, 2015 at 02:16:15AM +0200, Dhole wrote:
> On 08/31/2015 09:32 PM, Santiago Vila wrote:
> > Would be possible to drop the forked gettext and try to fix this in
> > the affected packages?
>
> Hi Santiago, I saw that you have been working on the packages affected
> by the POT creation date timestamps in gettext mo files today so maybe
> you know this already, but just to be sure:
> The patched gettext has been dropped from our reproducible APT
> repository, and all the affected packages by the issue
> different_pot_creation_date_in_gettext_mo_files (formerly
> lc_messages_randomness) have been rebuilt in our jenkins setup, using
> the gettext version from Debian unstable:
> https://reproducible.debian.net/issues/unstable/different_pot_creation_date_in_gettext_mo_files_issue.html
Thanks a lot.
Yesterday I realized that gettext has already some things to avoid
this problem. For example:
/usr/share/gettext/po/remove-potcdate.sin
/usr/share/gettext/po/Makefile.in.in
When I had more time I'll try to put this in the Wiki somewhere, but
as a short summary (feel free to take any of this and open the wiki
page yourself), I would say:
The .po files inside a Debian source package are supposed to be
"final" for that particular Debian package version and should not need
to be "refreshed", "updated" or "msgmerged" at build time.
(Note that this is only a "matter of principles", finding a fix for
the offending packages might be not always trivial).
Marked as found in versions gettext/0.19.7-2.
Request was from Alexis Bienvenüe <pado@passoire.fr>
to 825436-submit@bugs.debian.org.
(Thu, 26 May 2016 21:36:17 GMT) (full text, mbox, link).
Added tag(s) upstream.
Request was from Alexis Bienvenüe <pado@passoire.fr>
to 825436-submit@bugs.debian.org.
(Thu, 26 May 2016 21:36:18 GMT) (full text, mbox, link).
Merged 792687 825436
Request was from Alexis Bienvenüe <pado@passoire.fr>
to 825436-submit@bugs.debian.org.
(Thu, 26 May 2016 21:36:20 GMT) (full text, mbox, link).
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Sun, 20 Nov 2016 13:00:02 GMT) (full text, mbox, link).
Acknowledgement sent
to Bruno Haible <bruno@clisp.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Sun, 20 Nov 2016 13:00:02 GMT) (full text, mbox, link).
Message #51 received at 792687@bugs.debian.org (full text, mbox, reply):
A bug has been opened in the Gettext bug tracker about this issue:
https://savannah.gnu.org/bugs/?49654
In my opinion (as former Gettext maintainer) the fix ought to go into
msgfmt, not xgettext.
The gettext.diff.txt patch from
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792687
is therefore the wrong approach, IMO.
Bruno
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Sun, 02 Jul 2017 08:33:06 GMT) (full text, mbox, link).
Acknowledgement sent
to Daniel Shahaf <danielsh@apache.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Sun, 02 Jul 2017 08:33:06 GMT) (full text, mbox, link).
Message #56 received at 792687@bugs.debian.org (full text, mbox, reply):
Hi,
On Sun, Nov 20, 2016 at 01:57:48PM +0100, Bruno Haible wrote:
> In my opinion (as former Gettext maintainer) the fix ought to go into
> msgfmt, not xgettext.
>
> The gettext.diff.txt patch from
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792687
> is therefore the wrong approach, IMO.
Putting the fix into msgfmt is an option when the .mo file needs to be
reproducible, but I'm facing an issue where I'd like the .pot file
generated by xgettext to be reproducible as well.
I'm looking at a build process that generates a tar.gz from an svn tag.
The build process generates a .pot file from the tag and adds it to the
tar.gz. The .pot file embeds the current time, which makes the tar.gz
unreproducible. Changing msgfmt does not help in this situation, since
the .po→.mo compilation happens after the tar.gz is created.
One way to make the tar.gz reproducible would be for the build process
to tell xgettext what value to set the POT-Creation-Date to. Would that
be a good idea? If not, what would you suggest?
Cheers,
Daniel
P.S. I use xgettext from gettext-0.19.8.1-2 on stretch (but it's the
same version as in sid).
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Sat, 15 Jul 2017 17:45:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Bruno Haible <bruno@clisp.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Sat, 15 Jul 2017 17:45:03 GMT) (full text, mbox, link).
Message #61 received at 792687@bugs.debian.org (full text, mbox, reply):
Hello,
Daniel Shahaf wrote:
> Putting the fix into msgfmt is an option when the .mo file needs to be
> reproducible
Yes, .mo files are binary files, installed on the end users' systems. Therefore
we've made .mo files reproducible [1].
> but I'm facing an issue where I'd like the .pot file
> generated by xgettext to be reproducible as well.
>
> I'm looking at a build process that generates a tar.gz from an svn tag.
> The build process generates a .pot file from the tag and adds it to the
> tar.gz. The .pot file embeds the current time, which makes the tar.gz
> unreproducible. ...
> One way to make the tar.gz reproducible would be ...
There is no point in making a source tarball (.tar.gz) reproducible.
Read the first two paragraphs of [2].
1) A .pot file does not get installed on the end users' systems.
2) A .pot file is plain text, and the diffs between one version and another
version are of reasonable size.
For example, the recursive diff between libidn2-0.15 and libidn2-0.16 contains
this hunk:
diff -r -u libidn2-0.15/po/libidn2.pot libidn2-0.16/po/libidn2.pot
--- libidn2-0.15/po/libidn2.pot 2017-01-14 16:09:47.000000000 +0100
+++ libidn2-0.16/po/libidn2.pot 2017-01-16 09:13:10.000000000 +0100
@@ -6,9 +6,9 @@
#, fuzzy
msgid ""
msgstr ""
-"Project-Id-Version: libidn2 0.15\n"
+"Project-Id-Version: libidn2 0.16\n"
"Report-Msgid-Bugs-To: bug-libidn2@gnu.org\n"
-"POT-Creation-Date: 2017-01-14 16:09+0100\n"
+"POT-Creation-Date: 2017-01-16 09:13+0100\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
But SO WHAT? It does not get installed on the end user's system. it's plain
text, and it's a small diff.
Bruno
[1] http://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commit;h=d13f165b83701dffc14f7151419e0c00c00c0d1b
[2] https://reproducible-builds.org/
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Sat, 15 Jul 2017 18:09:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Daniel Shahaf <danielsh@apache.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Sat, 15 Jul 2017 18:09:03 GMT) (full text, mbox, link).
Message #66 received at 792687@bugs.debian.org (full text, mbox, reply):
Bruno Haible wrote on Sat, 15 Jul 2017 19:40 +0200:
> But SO WHAT? It does not get installed on the end user's system.
I am not disputing that .mo files, which are installed on user systems,
should be reproducible.
I am asserting that there is another workflow which would be simpler if
.po file headers were also reproducible.
> it's plain text, and it's a small diff.
This doesn't scale. (For example, in my use-case, I'm dealing with a
5000-line unified diff full of one-line changes in date strings and C
comments and any number of other things. My goal is to get the number of
lines down to zero.)
So: should po file generation allow the caller to control the timestamp
that would be embedded?
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Sat, 15 Jul 2017 18:21:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Santiago Vila <sanvila@unex.es>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Sat, 15 Jul 2017 18:21:05 GMT) (full text, mbox, link).
Message #71 received at 792687@bugs.debian.org (full text, mbox, reply):
On Sat, Jul 15, 2017 at 06:04:58PM +0000, Daniel Shahaf wrote:
> So: should po file generation allow the caller to control the timestamp
> that would be embedded?
Maybe, or maybe not.
But so far, the non-reproducible Debian packages using gettext I've
seen fail to be reproducible because maintainers insist on
regenerating a .pot file at build time and performing msgmerge on all
the .po files with the newly generated .pot file "just in case", or
doing similar things.
IMHO, that's clearly a task that would be better done before build time
(i.e. taken out from the build process), much like upstream authors call
autoconf before creating the final tarball when making a release.
Thanks.
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Sat, 15 Jul 2017 19:27:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Bruno Haible <bruno@clisp.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Sat, 15 Jul 2017 19:27:05 GMT) (full text, mbox, link).
Message #76 received at 792687@bugs.debian.org (full text, mbox, reply):
Daniel Shahaf wrote:
> > it's plain text, and it's a small diff.
>
> This doesn't scale. (For example, in my use-case, I'm dealing with a
> 5000-line unified diff full of one-line changes in date strings and C
> comments and any number of other things. My goal is to get the number of
> lines down to zero.)
Then you will have to filter out the one-line changes that are not
important to you.
The purpose of tarballs is distribution to developers, translators,
and distros.
The purpose of Makefile.in.in rules that generate .pot files is:
The .pot files are the starting point for translators (or for the
Translation Project, which extracts the .pot files and makes them
easily accessible to the translators).
The date of the POT file is important info for the translators.
This is the main workflow. It allows for reproducible builds (through
the timestamp filtering now built into msgfmt).
> So: should po file generation allow the caller to control the timestamp
> that would be embedded?
No, that would be a regression for the translators.
> I am asserting that there is another workflow which would be simpler if
> .po file headers were also reproducible.
You'll have to adapt. Accept the main workflow as it is.
Bruno
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Sat, 15 Jul 2017 19:39:06 GMT) (full text, mbox, link).
Acknowledgement sent
to Bruno Haible <bruno@clisp.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Sat, 15 Jul 2017 19:39:06 GMT) (full text, mbox, link).
Message #81 received at 792687@bugs.debian.org (full text, mbox, reply):
Santiago Vila wrote:
> But so far, the non-reproducible Debian packages using gettext I've
> seen fail to be reproducible because maintainers insist on
> regenerating a .pot file at build time and performing msgmerge on all
> the .po files with the newly generated .pot file "just in case", or
> doing similar things.
Have you included the support for reproducible builds [1] in the
Debian package of gettext?
If not, that's what you need to do. Debian and the GNU project are in
favour of reproducible builds, but no gettext release is planned for the next
few months. So please help out by pulling this patch.
"Regenerating a .pot file at build time" and "performing msgmerge on all
the .po files with the newly generated .pot file just in case" are not
a problem. They take a couple of CPU cycles, but they do not introduce
non-reproducibility - assuming you use patch [1].
If yes, then please show me a package that still is non-reproducible because
of gettext.
> IMHO, that's clearly a task that would be better done before build time
> (i.e. taken out from the build process), much like upstream authors call
> autoconf before creating the final tarball when making a release.
Different package maintainers do it differently, and there are good arguments
for doing it either way. There are plans to improve this situation [2].
Bruno
[1] http://git.savannah.gnu.org/gitweb/?p=gettext.git;a=commit;h=d13f165b83701dffc14f7151419e0c00c00c0d1b
[2] https://savannah.gnu.org/bugs/?50910
Information forwarded
to debian-bugs-dist@lists.debian.org, Santiago Vila <sanvila@debian.org>:
Bug#792687; Package src:gettext.
(Sat, 15 Jul 2017 20:39:02 GMT) (full text, mbox, link).
Acknowledgement sent
to Daniel Shahaf <danielsh@apache.org>:
Extra info received and forwarded to list. Copy sent to Santiago Vila <sanvila@debian.org>.
(Sat, 15 Jul 2017 20:39:02 GMT) (full text, mbox, link).
Message #86 received at 792687@bugs.debian.org (full text, mbox, reply):
Bruno Haible wrote on Sat, 15 Jul 2017 21:24 +0200:
> Daniel Shahaf wrote:
> > So: should po file generation allow the caller to control the timestamp
> > that would be embedded?
>
> No, that would be a regression for the translators.
>
No, it would not. The proposed change is to set the timestamp header to
a caller-provided value. Translators would not be providing a value,
and therefore would not exercise the codepath being proposed for
addition and would experience no change in behaviour.
> > > it's plain text, and it's a small diff.
> >
> > This doesn't scale. (For example, in my use-case, I'm dealing with a
> > 5000-line unified diff full of one-line changes in date strings and C
> > comments and any number of other things. My goal is to get the number of
> > lines down to zero.)
>
> Then you will have to filter out the one-line changes that are not
> important to you.
>
As I said, this doesn't scale. I have multiple upstreams to filter in
this way; I assume others are in my position too; and the filter would
be a moving part in itself.
To put it differently: if I have to use diffoscope to see that two
things are equivalent, I've already lost. There is a world of
difference between things that are equal according to cmp(1) and things
that are equal according to diffoscope(1): the latter notion of
equivalence is much weaker in practice. (E.g., it doesn't allow for
content addressing, it makes audits harder, etc.)
> The purpose of tarballs is distribution to developers, translators,
> and distros.
>
> The purpose of Makefile.in.in rules that generate .pot files is:
> The .pot files are the starting point for translators (or for the
> Translation Project, which extracts the .pot files and makes them
> easily accessible to the translators).
> The date of the POT file is important info for the translators.
>
The date would remain available to translators: whoever generates the
tarball for distribution to translators not be using the new codepath.
> This is the main workflow. It allows for reproducible builds (through
> the timestamp filtering now built into msgfmt).
>
> > I am asserting that there is another workflow which would be simpler if
> > .po file headers were also reproducible.
>
> You'll have to adapt. Accept the main workflow as it is.
You'll have to adapt. Keep the main workflow unperturbed but add a
modified workflow to cater for different use-cases.
tl;dr: The proposed change is 100% backwards compatible.
Daniel
(I'll slow down my replies for a bit to give others a chance to chime in.)
Reply sent
to Santiago Vila <sanvila@debian.org>:
You have taken responsibility.
(Sun, 20 Aug 2017 21:15:03 GMT) (full text, mbox, link).
Notification sent
to Dhole <dhole@openmailbox.org>:
Bug acknowledged by developer.
(Sun, 20 Aug 2017 21:15:03 GMT) (full text, mbox, link).
Message #91 received at 792687-close@bugs.debian.org (full text, mbox, reply):
Source: gettext
Source-Version: 0.19.8.1-3
We believe that the bug you reported is fixed in the latest version of
gettext, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 792687@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Santiago Vila <sanvila@debian.org> (supplier of updated gettext package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Sun, 20 Aug 2017 22:50:10 +0200
Source: gettext
Binary: gettext-base gettext gettext-el gettext-doc autopoint libgettextpo0 libasprintf0v5 libgettextpo-dev libasprintf-dev
Architecture: source
Version: 0.19.8.1-3
Distribution: unstable
Urgency: medium
Maintainer: Santiago Vila <sanvila@debian.org>
Changed-By: Santiago Vila <sanvila@debian.org>
Description:
autopoint - The autopoint program from GNU gettext
gettext - GNU Internationalization utilities
gettext-base - GNU Internationalization utilities for the base system
gettext-doc - Documentation for GNU gettext
gettext-el - Emacs po-mode for editing gettext .po files
libasprintf-dev - GNU Internationalization library development files
libasprintf0v5 - GNU library to use fprintf and friends in C++
libgettextpo-dev - GNU Internationalization library development files
libgettextpo0 - GNU Internationalization library
Closes: 792687 825436 871271
Changes:
gettext (0.19.8.1-3) unstable; urgency=medium
.
* Add g++ (>= 4:7) to Build-Depends to ensure it's built with GCC 7.
* Update shlibs file for libasprintf0v5. Closes: #871271.
* msgfmt: Remove POT-Creation-Date field from the header in the output.
Taken from upstream commit d13f165b83701dffc14f7151419e0c00c00c0d1b.
Should help reproducible builds. Closes: #792687, #825436.
* Drop spurious spaces/tabs in the changelog.
* Activate ldconfig via triggers.
* Build against libunistring2.
* Rename patches to *.patch.
Checksums-Sha1:
6ec2a03aef19705cf3295733331d8e24baf4e1a3 1975 gettext_0.19.8.1-3.dsc
8e4445cfd19c3fa198e4bae6124dd4d3a698404a 23624 gettext_0.19.8.1-3.debian.tar.xz
6d93070abf93e5db4529f733bdbe90111b887f0a 10169 gettext_0.19.8.1-3_source.buildinfo
Checksums-Sha256:
bc622d43e47f9df367856d184166495edaebbf894a03e0a9a3a05e2b7f655b64 1975 gettext_0.19.8.1-3.dsc
cbe851fd83714512df590e3155c91e3f33037c881c13f4e056a44bb131e463c0 23624 gettext_0.19.8.1-3.debian.tar.xz
e300d90a66848cbcda29655d1639d15bec22398f3f3b91ce432f05a0a5f5a836 10169 gettext_0.19.8.1-3_source.buildinfo
Files:
5ab0ae38f36c9c356dc573f463050b45 1975 devel optional gettext_0.19.8.1-3.dsc
74469034164910abee9835b3d8bff267 23624 devel optional gettext_0.19.8.1-3.debian.tar.xz
7d52ce253c75e3786b1834f366784cf1 10169 devel optional gettext_0.19.8.1-3_source.buildinfo
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEE1Uw7+v+wQt44LaXXQc5/C58bizIFAlmZ9lIACgkQQc5/C58b
izKX6Af+Kinl/z/dEqpQGXIG5BRppRwcCUqdtY20aErOFofmkj/0qFwjzMrly0u2
QWJMmbTKMuO689oZJYHojOvFodzRqOu3/KlGw+ge/ehkOGz0Swv80fVV/I3MtAkw
12n1NFogXTDCnvL4NlcWrIhLyhemPfzX/+6179r2sS2QjAvOFqJ+goF19WD1xhxl
3dSmev+eomySHN6iNGN67uNts7/tf76rgbUoZPf2J0WDNNE6Eanlg5jQfp7r6JwT
nmwwk5matF90/iSETqPcjVGKl9rUHY/IRc52sVqruPh22C9uVwEUTWM3zCNq86K4
KjlUEGccIgKQNz5RfItYk3b+00oGMQ==
=Do+B
-----END PGP SIGNATURE-----
Reply sent
to Santiago Vila <sanvila@debian.org>:
You have taken responsibility.
(Sun, 20 Aug 2017 21:15:04 GMT) (full text, mbox, link).
Notification sent
to Alexis Bienvenüe <pado@passoire.fr>:
Bug acknowledged by developer.
(Sun, 20 Aug 2017 21:15:04 GMT) (full text, mbox, link).
Message #96 received at 825436-close@bugs.debian.org (full text, mbox, reply):
Source: gettext
Source-Version: 0.19.8.1-3
We believe that the bug you reported is fixed in the latest version of
gettext, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 825436@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Santiago Vila <sanvila@debian.org> (supplier of updated gettext package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Sun, 20 Aug 2017 22:50:10 +0200
Source: gettext
Binary: gettext-base gettext gettext-el gettext-doc autopoint libgettextpo0 libasprintf0v5 libgettextpo-dev libasprintf-dev
Architecture: source
Version: 0.19.8.1-3
Distribution: unstable
Urgency: medium
Maintainer: Santiago Vila <sanvila@debian.org>
Changed-By: Santiago Vila <sanvila@debian.org>
Description:
autopoint - The autopoint program from GNU gettext
gettext - GNU Internationalization utilities
gettext-base - GNU Internationalization utilities for the base system
gettext-doc - Documentation for GNU gettext
gettext-el - Emacs po-mode for editing gettext .po files
libasprintf-dev - GNU Internationalization library development files
libasprintf0v5 - GNU library to use fprintf and friends in C++
libgettextpo-dev - GNU Internationalization library development files
libgettextpo0 - GNU Internationalization library
Closes: 792687 825436 871271
Changes:
gettext (0.19.8.1-3) unstable; urgency=medium
.
* Add g++ (>= 4:7) to Build-Depends to ensure it's built with GCC 7.
* Update shlibs file for libasprintf0v5. Closes: #871271.
* msgfmt: Remove POT-Creation-Date field from the header in the output.
Taken from upstream commit d13f165b83701dffc14f7151419e0c00c00c0d1b.
Should help reproducible builds. Closes: #792687, #825436.
* Drop spurious spaces/tabs in the changelog.
* Activate ldconfig via triggers.
* Build against libunistring2.
* Rename patches to *.patch.
Checksums-Sha1:
6ec2a03aef19705cf3295733331d8e24baf4e1a3 1975 gettext_0.19.8.1-3.dsc
8e4445cfd19c3fa198e4bae6124dd4d3a698404a 23624 gettext_0.19.8.1-3.debian.tar.xz
6d93070abf93e5db4529f733bdbe90111b887f0a 10169 gettext_0.19.8.1-3_source.buildinfo
Checksums-Sha256:
bc622d43e47f9df367856d184166495edaebbf894a03e0a9a3a05e2b7f655b64 1975 gettext_0.19.8.1-3.dsc
cbe851fd83714512df590e3155c91e3f33037c881c13f4e056a44bb131e463c0 23624 gettext_0.19.8.1-3.debian.tar.xz
e300d90a66848cbcda29655d1639d15bec22398f3f3b91ce432f05a0a5f5a836 10169 gettext_0.19.8.1-3_source.buildinfo
Files:
5ab0ae38f36c9c356dc573f463050b45 1975 devel optional gettext_0.19.8.1-3.dsc
74469034164910abee9835b3d8bff267 23624 devel optional gettext_0.19.8.1-3.debian.tar.xz
7d52ce253c75e3786b1834f366784cf1 10169 devel optional gettext_0.19.8.1-3_source.buildinfo
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEE1Uw7+v+wQt44LaXXQc5/C58bizIFAlmZ9lIACgkQQc5/C58b
izKX6Af+Kinl/z/dEqpQGXIG5BRppRwcCUqdtY20aErOFofmkj/0qFwjzMrly0u2
QWJMmbTKMuO689oZJYHojOvFodzRqOu3/KlGw+ge/ehkOGz0Swv80fVV/I3MtAkw
12n1NFogXTDCnvL4NlcWrIhLyhemPfzX/+6179r2sS2QjAvOFqJ+goF19WD1xhxl
3dSmev+eomySHN6iNGN67uNts7/tf76rgbUoZPf2J0WDNNE6Eanlg5jQfp7r6JwT
nmwwk5matF90/iSETqPcjVGKl9rUHY/IRc52sVqruPh22C9uVwEUTWM3zCNq86K4
KjlUEGccIgKQNz5RfItYk3b+00oGMQ==
=Do+B
-----END PGP SIGNATURE-----
Bug archived.
Request was from Debbugs Internal Request <owner@bugs.debian.org>
to internal_control@bugs.debian.org.
(Sun, 24 Sep 2017 07:27:12 GMT) (full text, mbox, link).
Send a report that this bug log contains spam.
Debian bug tracking system administrator <owner@bugs.debian.org>.
Last modified:
Sat Aug 19 15:06:08 2023;
Machine Name:
buxtehude
Debian Bug tracking system
Debbugs is free software and licensed under the terms of the GNU
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.
Copyright © 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson,
2005-2017 Don Armstrong, and many other contributors.