Debian Bug report logs - #1004950
reprotest fails when using kk_KZ.RK1048 locale with java (locale not supported by java)

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Fab Stz <fabstz-it@yahoo.fr>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: reprotest fails when using kk_KZ.RK1048 locale with java (locale not supported by java)

Date: Fri, 04 Feb 2022 08:50:46 +0100

Package: reprotest
Version: 0.7.16
Severity: normal

Dear Maintainer,

I have a CI job that runs reprotest. The project, at some time, makes calls to
java. However this fails/crashes sometimes.

By narrowing things down it always crashes when locale kk_KZ.RK1048 is in use.

By searching further I discovered that kk_KZ.RK1048 locale/charset is not 
supported by java [1]. Hence the crash:

$ LC_ALL=kk_KZ.RK1048 java -version
Error occurred during initialization of VM
java.lang.IllegalArgumentException: Null charset name
        at java.nio.charset.Charset.lookup(java.base/Charset.java:455)
        at java.nio.charset.Charset.defaultCharset(java.base/Charset.java:608)
        at java.lang.StringCoding.decode(java.base/StringCoding.java:316)
        at java.lang.String.<init>(java.base/String.java:592)
        at java.lang.String.<init>(java.base/String.java:614)
        at java.lang.System.initProperties(java.base/Native Method)
        at java.lang.System.initPhase1(java.base/System.java:1948)



$ LC_ALL=C java -version
openjdk version "11.0.12" 2021-07-20
OpenJDK Runtime Environment (build 11.0.12+7-post-Debian-2)
OpenJDK 64-Bit Server VM (build 11.0.12+7-post-Debian-2, mixed mode, sharing)


Expected behavior? That it works also with call to java? or maybe an option to
disable this locale if java is used?

[1]:
https://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html

-- System Information:
Debian Release: 11.2
  APT prefers stable-updates
  APT policy: (991, 'stable-updates'), (991, 'stable'), (95, 'testing'), (90,
'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.10.0-10-amd64 (SMP w/4 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE,
TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8),
LANGUAGE=fr:en_US
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages reprotest depends on:
ii  apt-utils              2.2.4
pn  diffoscope             <none>
ii  libdpkg-perl           1.20.9
ii  procps                 2:3.3.17-5
ii  python3                3.9.2-3
ii  python3-debian         0.1.39
ii  python3-distro         1.5.0-1
ii  python3-pkg-resources  52.0.0-4
pn  python3-rstr           <none>

Versions of packages reprotest recommends:
pn  diffoscope          <none>
pn  diffoscope-minimal  <none>
pn  disorderfs          <none>
pn  faketime            <none>
pn  locales-all         <none>
ii  sudo                1.9.5p2-3

Versions of packages reprotest suggests:
ii  autodep8     0.24
pn  qemu-system  <none>
ii  qemu-utils   1:5.2+dfsg-11+deb11u1

Message #10 received at 1004950@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@reproducible-builds.org>

To: Fab Stz <fabstz-it@yahoo.fr>, 1004950@bugs.debian.org

Subject: Re: Bug#1004950: reprotest fails when using kk_KZ.RK1048 locale with java (locale not supported by java)

Date: Fri, 04 Feb 2022 09:18:56 -0800

[Message part 1 (text/plain, inline)]

On 2022-02-04, Fab Stz wrote:
> I have a CI job that runs reprotest. The project, at some time, makes calls to
> java. However this fails/crashes sometimes.
>
> By narrowing things down it always crashes when locale kk_KZ.RK1048 is in use.
>
> By searching further I discovered that kk_KZ.RK1048 locale/charset is not 
> supported by java [1]. Hence the crash:
>
> $ LC_ALL=kk_KZ.RK1048 java -version
> Error occurred during initialization of VM
> java.lang.IllegalArgumentException: Null charset name
>         at java.nio.charset.Charset.lookup(java.base/Charset.java:455)
...
> $ LC_ALL=C java -version
> openjdk version "11.0.12" 2021-07-20
> OpenJDK Runtime Environment (build 11.0.12+7-post-Debian-2)
> OpenJDK 64-Bit Server VM (build 11.0.12+7-post-Debian-2, mixed mode, sharing)
>
>
> Expected behavior? That it works also with call to java? or maybe an option to
> disable this locale if java is used?
>
> [1]:
> https://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html

Well, reprotest intentionally uses some obscure locales to find bugs
exactly like this...

That said, this points out two issues, and I'm considering making some
changes to locale handling for the "experiment" tests. Currently:

  loc = random.choice(['fr_CH.UTF-8', 'es_ES', 'ru_RU.CP1251', 'kk_KZ.RK1048', 'zh_CN'])

I don't think selecting a locale at random is a good idea; this means
sometimes a build might succeed with reprotest and sometimes not,
depending on which locale happens to be randomly selected.

Secondly, I think by default reprotest should use a slightly less
obscure locale to test, and the same locale every time.

Adding a commandline flag to build with a specified locale would also be
good. Possibly another flag to do a series of builds, each with a
different locale (maybe from a list of very obscure locales).


live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Message #15 received at 1004950@bugs.debian.org (full text, mbox, reply):

From: Holger Levsen <holger@layer-acht.org>

To: Vagrant Cascadian <vagrant@reproducible-builds.org>, 1004950@bugs.debian.org

Cc: Fab Stz <fabstz-it@yahoo.fr>

Subject: Re: Bug#1004950: reprotest fails when using kk_KZ.RK1048 locale with java (locale not supported by java)

Date: Fri, 4 Feb 2022 17:59:27 +0000

[Message part 1 (text/plain, inline)]

On Fri, Feb 04, 2022 at 09:18:56AM -0800, Vagrant Cascadian wrote:
> I don't think selecting a locale at random is a good idea; this means
> sometimes a build might succeed with reprotest and sometimes not,
> depending on which locale happens to be randomly selected.

agreed.

> Secondly, I think by default reprotest should use a slightly less
> obscure locale to test, and the same locale every time.

also agreed.

> Adding a commandline flag to build with a specified locale would also be
> good. Possibly another flag to do a series of builds, each with a
> different locale (maybe from a list of very obscure locales).

this seems like a good idea indeed.


-- 
cheers,
	Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁  holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀  OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C
 ⠈⠳⣄

Imagine god created trillions of galaxies but freaks out because some dude
kisses another.

[signature.asc (application/pgp-signature, inline)]

Message #18 received at 1004950-submitter@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <noreply@salsa.debian.org>

To: 1004950-submitter@bugs.debian.org

Subject: Bug#1004950 marked as pending in reprotest

Date: Thu, 16 Feb 2023 22:33:17 +0000

Control: tag -1 pending

Hello,

Bug #1004950 in reprotest reported by you has been fixed in the
Git repository and is awaiting an upload. You can see the commit
message below and you can check the diff of the fix at:

https://salsa.debian.org/reproducible-builds/reprotest/-/commit/610e6cae15fe8d066303694af97bf3a09a6b01c2

------------------------------------------------------------------------
Do not randomize chosen locale and use a UTF-8 supported locale.
(Closes: #925879, #1004950)

Use an Estonian locale, which has unusual sort order compared to many
european languages, which may catch some interesting issues.

Testing more obscure locales and character sets is certainly
desireable, but it is also very important for reprotest to pick the
varied locale deterministically.

Randomly choosing the locale, or choosing based on a hash of the
inputs, might lead to picking a different locale while debugging
patches for a potential fix, with misleading results if the issue is
highly locale specific.
------------------------------------------------------------------------

(this message was generated automatically)
-- 
Greetings

https://bugs.debian.org/1004950

Message #25 received at 1004950-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 1004950-close@bugs.debian.org

Subject: Bug#1004950: fixed in reprotest 0.7.23

Date: Mon, 20 Feb 2023 12:19:33 +0000

Source: reprotest
Source-Version: 0.7.23
Done: Holger Levsen <holger@debian.org>

We believe that the bug you reported is fixed in the latest version of
reprotest, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 1004950@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Holger Levsen <holger@debian.org> (supplier of updated reprotest package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Mon, 20 Feb 2023 12:55:54 +0100
Source: reprotest
Architecture: source
Version: 0.7.23
Distribution: unstable
Urgency: medium
Maintainer: Reproducible builds folks <reproducible-builds@lists.alioth.debian.org>
Changed-By: Holger Levsen <holger@debian.org>
Closes: 925879 1004950
Changes:
 reprotest (0.7.23) unstable; urgency=medium
 .
   [ Vagrant Cascadian ]
   * Do not randomize chosen locale and always vary with the same UTF-8
     supported locale., et_EE.UTF-8. (Closes: #925879, #1004950)
   * Support passing --vary=locales.locale=LOCALE to specify locale to vary.
   * Document the above in README.
 .
   [ Holger Levsen ]
   * README:
     - improve examples how to build a Debian package.
     - mention interesting locales in README.
   * Drop d/NEWS as it only contained one entry from 2017.
 .
   [ lintian-brush ]
   * Bump standards version to 4.6.2, no changes needed.
   * Fill in Homepage field.
Checksums-Sha1:
 6334544444fa91a478bb1fb33a419f45b614d602 2241 reprotest_0.7.23.dsc
 91f798de816140b9db8e89a5439b60c77bec7956 88696 reprotest_0.7.23.tar.xz
 4a040b3da7aef4aface956f48818215531df206f 12296 reprotest_0.7.23_source.buildinfo
Checksums-Sha256:
 20375e6d63f933190425e95ea22b18cf09decb9d2ad5e0070c1ee04e3f659a6d 2241 reprotest_0.7.23.dsc
 3f7390ab44e0f8757db52e24c09a6721a252642d950a2b8210bcc31b17d95317 88696 reprotest_0.7.23.tar.xz
 bc42fcf307c8fb6ad304cce6a9d7f3903ae954d1a39c1832d92bcadd5dc62ad4 12296 reprotest_0.7.23_source.buildinfo
Files:
 d1365a077d7f43aeb3ef80910afa6995 2241 devel optional reprotest_0.7.23.dsc
 36901665f9e876ec62ddef47e7b67187 88696 devel optional reprotest_0.7.23.tar.xz
 a349a6c2d0cbceb1a1059151a9a91432 12296 devel optional reprotest_0.7.23_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEuL9UE3sJ01zwJv6dCRq4VgaaqhwFAmPzYMkACgkQCRq4Vgaa
qhzOJA//YpGxZgg1C2lledEHA7O1FdheLRyp+FfZV4uPM2MfCGbNknQxuVO2IVac
86spu/8hwFEzr0dNBi72zisJBkNNtXrtZDIAE5LaNx86h+qX6tQD4E5/V7IuYNEV
RMMymyqrVw865DiErmnVhuZA5Stq9M3mtc1TcnO3vYKDmEnwwxLDLcSM0vILKEnB
Khx9XJlNI4fmCzEyMeGydChWIEgFRrHRgixDwXtK1dxfMwEhuot7SsB7V0PJtg/p
1mreD1pX2xtQOTgGrMUPYbVPeU18I2G3onyG6yD9NzJBqARtitzPJ3rwEbWFVcqT
kgAnL2CdC28KyuHwKDx4/vxoJwVpxCHnY6sXrirYnTQwmnY8Rm56lOh3z1ixUtK4
vCrmxsiB3j6iuQrwPTgEFkkpC0nq5JKAZHOvjLeKGc7FZD7WvwW0W1eXAppCLdEm
TiI/NQUi62Z4hCy3KBVnvCIMu+lcKnHcMxFNz9QJZyrPcqc15DqWSqtalpfJc1Nr
8WV0CDcZUUrMEG8ahybdmNfEg/QHhoRGD8bnxtHFJ5RGOrFqqDzYPXjLSiGaJl/Z
UDGNJEgLcLax6xgS6rjyhmfOABR3igKXVhm7bwZcNtdglwEJ3N2w9FZrFd+e8SBW
lfbhKN8/iA8GY3EGmdeuMfjggvctbK2SuV10yXXOpc9M/d22Klw=
=4BKs
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.