Debian Bug report logs - #973414
rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Reply or subscribe to this bug.

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Fri, 30 Oct 2020 10:35:21 +0200

Package: libmozjs-78-0
Version: 78.3.0-2
Severity: important

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

[  165.903916] traps: gnome-shell[869] trap invalid opcode ip:b5518f8a sp:b17d6d80 error:0 in libmozjs-78.so.78.3.0[b4b98000+98c000]
[  186.163963] traps: gnome-shell[931] trap invalid opcode ip:b55d4f8a sp:b1892d80 error:0 in libmozjs-78.so.78.3.0[b4c54000+98c000]
[  188.417729] traps: gnome-shell[936] trap invalid opcode ip:b554df8a sp:b180bd80 error:0 in libmozjs-78.so.78.3.0[b4bcd000+98c000]
[  204.840919] traps: gnome-session-f[939] trap invalid opcode ip:b4e7f86a sp:bfcc1460 error:0 in librsvg-2.so.2.47.0[b4e6d000+5b7000]
[  205.334962] rfkill: input handler disabled
[  206.914176] traps: gjs[1056] trap invalid opcode ip:b74f9f8a sp:b3f6d040 error:0 in libmozjs-78.so.78.3.0[b6b79000+98c000]
[  217.300333] rfkill: input handler enabled
[  233.628071] traps: gnome-shell[1132] trap invalid opcode ip:b55c2f8a sp:b1880d80 error:0 in libmozjs-78.so.78.3.0[b4c42000+98c000]
[  237.282981] traps: gnome-shell[1145] trap invalid opcode ip:b554bf8a sp:b1809d80 error:0 in libmozjs-78.so.78.3.0[b4bcb000+98c000]
[  251.475717] traps: gnome-session-f[1148] trap invalid opcode ip:b4e5186a sp:bff460f0 error:0 in librsvg-2.so.2.47.0[b4e3f000+5b7000]
[  252.530390] rfkill: input handler disabled
[  252.935696] traps: gjs[1252] trap invalid opcode ip:b752cf8a sp:b3fa0040 error:0 in libmozjs-78.so.78.3.0[b6bac000+98c000]
[  257.741066] rfkill: input handler enabled
[  273.153629] traps: gnome-shell[1319] trap invalid opcode ip:b5570f8a sp:b182ed80 error:0 in libmozjs-78.so.78.3.0[b4bf0000+98c000]


- -- System Information: Debian Release: bullseye/sid
  APT prefers testing-debug
  APT policy: (1000, 'testing-debug'), (1000, 'testing'), (500, 'stable')
Architecture: i386 (i586)

Kernel: Linux 5.9.0-1-686 (SMP w/1 CPU thread)
Locale: LANG=fi_FI.UTF-8, LC_CTYPE=fi_FI.UTF-8 (charmap=UTF-8), LANGUAGE=fi:en
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages libmozjs-78-0 depends on:
ii  libc6       2.31-4
ii  libgcc-s1   10.2.0-15
ii  libicu67    67.1-4
ii  libstdc++6  10.2.0-15
ii  tzdata      2020d-1
ii  zlib1g      1:1.2.11.dfsg-2

libmozjs-78-0 recommends no packages.

libmozjs-78-0 suggests no packages.

- -- no debconf information

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEyJACx3qL7GpObXOQrh+Cd8S017YFAl+b0EIACgkQrh+Cd8S0
17a2QhAAhlMV2BrAoPmp2rCI/BUU5KnSQ+qlSFDz4Kd8to6RA+dwD5nzPcJAoBVK
IueznUCRvFMke8J7bJWnmZfOWnhA+7NGfl4sPitKD9b5V1Qe2TK5sxt16o5RvrR/
MIOEL9eJSgpToGZisToi+zKwbTmqLz4QuI9p4L6Fr27+sHVUpoyts7aFdsh2q9IL
XowB36SzfFxg8Xfu0AA6EilPqV54j/WGUHvI7MJH+gAOCDK9FkijsoxiwWKtanRe
DhjLrlelR+LgBONRN5LlcFHfHcd1pJJdMVtyt6mHMpc0FcY7QUb48ke5o2LBExb4
DAVmJBz/5zSezv+S6kpcgt2EJwgnGs6jkNiYjXQJj+NOypm0M2NiGovV2+OuuMzL
pbwrN/MjM8X6YClmUB1YYVrXUEyH3Leu3vILZ/w6sxN1Qgf6QVFfMVYoSwH8K4dh
NEUZkYKhEAsX0HBSaNvoUFhC0X8hH3NEmeSmwsb5vLcwZ8NEPMwS9efhnoaqWbzZ
vH3PHf4P1d+aAVPrLBU1ZGRGJoWooAzvNb7JUMBdt3tmKveXSetNxSRi1qa8kMRb
38dF3g7Z+kLoY3a4Sv6syi30YDeSvFd4VdzV/JjVvL+BnRRW77zUsyhFTuO2D3hc
l6st16Il04XEAKrp92sHsv8Rk0cf5X4vJJ5GPSo0iWCxacPhYU4=
=rnps
-----END PGP SIGNATURE-----

Message #10 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Martin-Éric Racine <martin-eric.racine@iki.fi>, 973414@bugs.debian.org

Cc: debian-gcc@lists.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Fri, 30 Oct 2020 11:49:57 +0000

Control: tags -1 + moreinfo

On Fri, 30 Oct 2020 at 10:35:21 +0200, Martin-Éric Racine wrote:
> [  165.903916] traps: gnome-shell[869] trap invalid opcode ip:b5518f8a sp:b17d6d80 error:0 in libmozjs-78.so.78.3.0[b4b98000+98c000]

Which specific x86 CPU is this? I see `uname -m` is i586 rather than the
expected i686 (and there's only one core), so presumably it's somewhere
close to our architecture baseline, either above or below.  The baseline
is -march=i686 since gcc-6_6.1.1-1 and stretch, according to the research
I did for <https://wiki.debian.org/ArchitectureSpecificsMemo>.

The /proc/cpuinfo would be useful information, since that includes a list
of capability flags.

cc gcc maintainers (in the absence of an i386 porter contact): Is there
a more formal/detailed specification for what CPU extensions we do and
don't intend to require on i386?

If this CPU is at or slightly above the baseline and mozjs78 is using
non-baseline instructions, then that's a bug in the mozjs78 source (most
likely the JIT) or the packaging. If this is the case, I suspect that our
upstream (Mozilla) will refuse to support older CPUs, so we will have to
choose between making non-upstreamable changes or declaring that mozjs78
violating the baseline is wontfix. Our i386 mozjs78 is already built in a
configuration that upstream don't support, because they assume SSE on i386,
but SSE isn't part of our baseline, so we are still using the legacy i387
FPU instructions and their weird extended-precision behaviour; some tests
fail and are skipped as a result.

If it's slightly below the baseline (only a few missing opcodes), then it
isn't a bug for packages to not work, but it's possible that you get away
with it most of the time because the missing opcodes are rarely emitted?

Does Firefox work on this machine?

Did older versions of GNOME Shell (buster or stretch) work on this machine?
If yes, do they have acceptable performance in practice?

> [  204.840919] traps: gnome-session-f[939] trap invalid opcode ip:b4e7f86a sp:bfcc1460 error:0 in librsvg-2.so.2.47.0[b4e6d000+5b7000]

If this is considered a mozjs78 bug, then librsvg has a similar bug
(possibly related to Rust, which they both use); or if your CPU is just
too old to run binaries compiled with Debian's post-stretch assumptions,
then the same applies to librsvg.

librsvg is likely to be more of a problem than mozjs78, because you can
take mozjs78 out of the critical path by using XFCE, LXDE or one of the
GNOME 2 forks like MATE and GNOME Flashback, but if this is a desktop
system you are still going to want to render SVGs.

    smcv

Message #17 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, debian-gcc@lists.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Fri, 30 Oct 2020 14:07:17 +0200

pe 30. lokak. 2020 klo 13.50 Simon McVittie (smcv@debian.org) kirjoitti:
> On Fri, 30 Oct 2020 at 10:35:21 +0200, Martin-Éric Racine wrote:
> > [  165.903916] traps: gnome-shell[869] trap invalid opcode ip:b5518f8a sp:b17d6d80 error:0 in libmozjs-78.so.78.3.0[b4b98000+98c000]
>
> Which specific x86 CPU is this? I see `uname -m` is i586 rather than the
> expected i686 (and there's only one core), so presumably it's somewhere
> close to our architecture baseline, either above or below.  The baseline
> is -march=i686 since gcc-6_6.1.1-1 and stretch, according to the research
> I did for <https://wiki.debian.org/ArchitectureSpecificsMemo>.

Geode LX800. This is technically a 686 but it doesn't support PAE, so
the CPU family remains 586.

> The /proc/cpuinfo would be useful information, since that includes a list
> of capability flags.

processor    : 0
vendor_id    : AuthenticAMD
cpu family    : 5
model        : 10
model name    : Geode(TM) Integrated Processor by AMD PCS
stepping    : 2
cpu MHz        : 498.025
cache size    : 128 KB
physical id    : 0
siblings    : 1
core id        : 0
cpu cores    : 1
apicid        : 0
initial apicid    : 0
fdiv_bug    : no
f00f_bug    : no
coma_bug    : no
fpu        : yes
fpu_exception    : yes
cpuid level    : 1
wp        : yes
flags        : fpu de pse tsc msr cx8 sep pge cmov clflush mmx mmxext
3dnowext 3dnow cpuid 3dnowprefetch vmmcall
bugs        : sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass
bogomips    : 996.05
clflush size    : 32
cache_alignment    : 32
address sizes    : 32 bits physical, 32 bits virtual
power management:

> cc gcc maintainers (in the absence of an i386 porter contact): Is there
> a more formal/detailed specification for what CPU extensions we do and
> don't intend to require on i386?

AFAIK plain 686 (without PAE) is the baseline. Debian's
linux-image-686 actually is configured for Geode.

> Does Firefox work on this machine?
>
> Did older versions of GNOME Shell (buster or stretch) work on this machine?
> If yes, do they have acceptable performance in practice?

Both have worked until whatever is currently in Stable. Firefox as-is,
gnome-shell using the X wrapper.

Martin-Éric

Message #22 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Martin-Éric Racine <martin-eric.racine@iki.fi>, 973414@bugs.debian.org

Cc: rustc@packages.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Sun, 8 Nov 2020 12:39:50 +0000

Control: retitle -1 libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode
Control: tags -1 = help

On Fri, 30 Oct 2020 at 14:07:17 +0200, Martin-Éric Racine wrote:
> pe 30. lokak. 2020 klo 13.50 Simon McVittie (smcv@debian.org) kirjoitti:
> > On Fri, 30 Oct 2020 at 10:35:21 +0200, Martin-Éric Racine wrote:
> > > [  165.903916] traps: gnome-shell[869] trap invalid opcode ip:b5518f8a sp:b17d6d80 error:0 in libmozjs-78.so.78.3.0[b4b98000+98c000]
> >
> > Which specific x86 CPU is this?
> 
> Geode LX800. This is technically a 686 but it doesn't support PAE, so
> the CPU family remains 586.

> > The /proc/cpuinfo would be useful information, since that includes a list
> > of capability flags.
> flags        : fpu de pse tsc msr cx8 sep pge cmov clflush mmx mmxext
> 3dnowext 3dnow cpuid 3dnowprefetch vmmcall

Notably, this doesn't include SSE (any version), or various smaller
extensions like POPCNT.

My guess would be that there's a hidden SSE assumption somewhere
in mozjs. I'm fairly sure Mozilla upstream doesn't really support
non-SSE CPUs any more (several tests fail because we're using i387
rather than SSE for arithmetic), so any porting to older x86 CPUs would
be Debian-specific. The Debian GNOME team packages mozjs because we need
it for GNOME Shell and gjs, but we don't really have the knowledge or
resources to port it to (sub)architectures that don't meet upstream's
assumptions - any help you can provide would be appreciated.

Because your syslog also included similar crashes in librsvg, which is
unrelated to mozjs except that both involve Rust code, I wonder whether
it might be the rust compiler rather than mozjs' JIT that is emitting
non-i586 opcodes on x86?

    smcv

Message #31 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Mon, 9 Nov 2020 20:05:01 +0200

su 8. marrask. 2020 klo 14.40 Simon McVittie (smcv@debian.org) kirjoitti:
> Because your syslog also included similar crashes in librsvg, which is
> unrelated to mozjs except that both involve Rust code, I wonder whether
> it might be the rust compiler rather than mozjs' JIT that is emitting
> non-i586 opcodes on x86?

Good observation. I'd really like to know this as well.

Worse comes, if it indeed turns out that libmozjs doesn't support
non-PAE hosts anymore, I could easily switch to lightdm or lxdm.
However, without librsvg, pretty much all display managers break,
since they all use the default SVG background configured by
desktop-base.

Martin-Éric

Message #36 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Thu, 3 Dec 2020 22:55:40 +0200

ma 9. marrask. 2020 klo 20.05 Martin-Éric Racine
(martin-eric.racine@iki.fi) kirjoitti:
>
> su 8. marrask. 2020 klo 14.40 Simon McVittie (smcv@debian.org) kirjoitti:
> > Because your syslog also included similar crashes in librsvg, which is
> > unrelated to mozjs except that both involve Rust code, I wonder whether
> > it might be the rust compiler rather than mozjs' JIT that is emitting
> > non-i586 opcodes on x86?
>
> Good observation. I'd really like to know this as well.

Seems that something broke there a while back. See bug #891561.

Martin-Éric

Message #41 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Tue, 8 Dec 2020 17:35:24 +0200

ma 9. marrask. 2020 klo 20.05 Martin-Éric Racine
(martin-eric.racine@iki.fi) kirjoitti:
>
> su 8. marrask. 2020 klo 14.40 Simon McVittie (smcv@debian.org) kirjoitti:
> > Because your syslog also included similar crashes in librsvg, which is
> > unrelated to mozjs except that both involve Rust code, I wonder whether
> > it might be the rust compiler rather than mozjs' JIT that is emitting
> > non-i586 opcodes on x86?
>
> Good observation. I'd really like to know this as well.
>
> Worse comes, if it indeed turns out that libmozjs doesn't support
> non-PAE hosts anymore, I could easily switch to lightdm or lxdm.
> However, without librsvg, pretty much all display managers break,
> since they all use the default SVG background configured by
> desktop-base.

Having downgraded and pinned all librsvg2 packages to what's in
oldstable (2.40.16) restores GDM's fallback to Xorg at least enough to
display gnome-session-failed. I suspect that the only bit of failure
that's left is because of mozjs, which unfortunately cannot be
downgraded, because it's a versioned dependency for a number of
packages. This would suggest that the problem indeed is with the Rust
compiler.

I'd really love to hear from Rust's maintainer at this point.

Martin-Éric

Message #46 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Thu, 10 Dec 2020 18:29:39 +0200

ti 8. jouluk. 2020 klo 17.35 Martin-Éric Racine
(martin-eric.racine@iki.fi) kirjoitti:
>
> ma 9. marrask. 2020 klo 20.05 Martin-Éric Racine
> (martin-eric.racine@iki.fi) kirjoitti:
> >
> > su 8. marrask. 2020 klo 14.40 Simon McVittie (smcv@debian.org) kirjoitti:
> > > Because your syslog also included similar crashes in librsvg, which is
> > > unrelated to mozjs except that both involve Rust code, I wonder whether
> > > it might be the rust compiler rather than mozjs' JIT that is emitting
> > > non-i586 opcodes on x86?
> >
> > Good observation. I'd really like to know this as well.
> >
> > Worse comes, if it indeed turns out that libmozjs doesn't support
> > non-PAE hosts anymore, I could easily switch to lightdm or lxdm.
> > However, without librsvg, pretty much all display managers break,
> > since they all use the default SVG background configured by
> > desktop-base.
>
> Having downgraded and pinned all librsvg2 packages to what's in
> oldstable (2.40.16) restores GDM's fallback to Xorg at least enough to
> display gnome-session-failed. I suspect that the only bit of failure
> that's left is because of mozjs, which unfortunately cannot be
> downgraded, because it's a versioned dependency for a number of
> packages. This would suggest that the problem indeed is with the Rust
> compiler.
>
> I'd really love to hear from Rust's maintainer at this point.

Btw, when it comes to libmozjs, is there any attachment you'd need?
gdb backtrace, etc.?

Martin-Éric

Message #51 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Martin-Éric Racine <martin-eric.racine@iki.fi>, 973414@bugs.debian.org

Cc: rustc@packages.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Thu, 10 Dec 2020 17:47:46 +0000

On Thu, 10 Dec 2020 at 18:29:39 +0200, Martin-Éric Racine wrote:
> Btw, when it comes to libmozjs, is there any attachment you'd need?
> gdb backtrace, etc.?

A backtrace is always useful, and it would also be useful to know which
opcode not supported by your CPU it's trying to execute.

I'm sure you know more than I do about the finer points of the x86
instruction set and how opcodes map to CPU flags and architecture
extensions.

    smcv

Message #56 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes in libmozjs when launching GDM3

Date: Thu, 10 Dec 2020 20:02:42 +0200

to 10. jouluk. 2020 klo 19.47 Simon McVittie (smcv@debian.org) kirjoitti:
>
> On Thu, 10 Dec 2020 at 18:29:39 +0200, Martin-Éric Racine wrote:
> > Btw, when it comes to libmozjs, is there any attachment you'd need?
> > gdb backtrace, etc.?
>
> A backtrace is always useful, and it would also be useful to know which
> opcode not supported by your CPU it's trying to execute.

$ sudo coredumpctl debug 855 --output /tmp/coredump_gnome-shell
           PID: 855 (gnome-shell)
           UID: 101 (Debian-gdm)
           GID: 122 (Debian-gdm)
        Signal: 4 (ILL)
     Timestamp: Thu 2020-12-10 18:38:09 EET (1h 15min ago)
  Command Line: /usr/bin/gnome-shell
    Executable: /usr/bin/gnome-shell
 Control Group: /user.slice/user-101.slice/session-c2.scope
          Unit: session-c2.scope
         Slice: user-101.slice
       Session: c2
     Owner UID: 101 (Debian-gdm)
       Boot ID: b115ec37ecca4f75acd8d60bc6a44403
    Machine ID: 1063a9d1fb9df6e371ea9f94491345ed
      Hostname: geode
       Storage:
/var/lib/systemd/coredump/core.gnome-shell.101.b115ec37ecca4f75acd8d60bc6a44403.855.1607618289000000.zst
       Message: Process 855 (gnome-shell) of user 101 dumped core.

                Stack trace of thread 861:
                #0  0x00000000b5500d08
_ZN17compiler_builtins3int4udiv12__udivmoddi417h8768e341ea5a27afE
(libmozjs-78.so.0 + 0xa01d08)

GNU gdb (Debian 10.1-1+b1) 10.1
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/gnome-shell...
(No debugging symbols found in /usr/bin/gnome-shell)
[New LWP 861]
[New LWP 855]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".

warning: Can't read data for section '.debug_info' in file
'/usr/lib/debug/.build-id/8e/1bdec525caa9d999e8d03f4e90d5e350bddf4f.debug'
Core was generated by `/usr/bin/gnome-shell'.
Program terminated with signal SIGILL, Illegal instruction.
#0  0xb5500d08 in compiler_builtins::int::udiv::__udivmoddi4 () from
/usr/lib/i386-linux-gnu/libmozjs-78.so.0
[Current thread is 1 (Thread 0xb15c9b40 (LWP 861))]
(gdb) bt full
#0  0xb5500d08 in compiler_builtins::int::udiv::__udivmoddi4 () from
/usr/lib/i386-linux-gnu/libmozjs-78.so.0
No symbol table info available.
(gdb) c
The program is not being run.
(gdb)

> I'm sure you know more than I do about the finer points of the x86
> instruction set and how opcodes map to CPU flags and architecture
> extensions.

Sadly not. I only remember that the Geode LX is 1 instruction short of
a standard 686.

Martin-Éric

Message #61 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Martin-Éric Racine <martin-eric.racine@iki.fi>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org, 976374@bugs.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Thu, 10 Dec 2020 19:21:15 +0000

Control: reassign 973414 rustc
Control: reassign 976374 rustc
Control: forcemerge 973414 976374
Control: retitle 973414 rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386
Control: affects 973414 + libmozjs-78-0 src:mozjs78 librsvg2-2 src:librsvg

On Thu, 10 Dec 2020 at 20:02:42 +0200, Martin-Éric Racine wrote:
> Core was generated by `/usr/bin/gnome-shell'.
> Program terminated with signal SIGILL, Illegal instruction.
> #0  0xb5500d08 in compiler_builtins::int::udiv::__udivmoddi4 () from
> /usr/lib/i386-linux-gnu/libmozjs-78.so.0
> [Current thread is 1 (Thread 0xb15c9b40 (LWP 861))]
> (gdb) bt full
> #0  0xb5500d08 in compiler_builtins::int::udiv::__udivmoddi4 () from
> /usr/lib/i386-linux-gnu/libmozjs-78.so.0
> No symbol table info available.

This looks like Rust code, presumably compiled from something like
https://sources.debian.org/src/rustc/1.48.0+dfsg1-1/vendor/compiler_builtins/src/int/udiv.rs/

Debian's rustc has a patch to reduce the i386 baseline from upstream's
pentium4 to pentiumpro
https://sources.debian.org/src/rustc/1.48.0+dfsg1-1/debian/patches/d-i686-baseline.patch/
but apparently that's not sufficient for a Geode LX. i686 is in a weird
situation where the Pentium Pro was the *first* i686 CPU, but is not a
*baseline* i686 CPU.

Presumably the root cause of #976374 in librsvg, which also contains
Rust code, is the same.

> > I'm sure you know more than I do about the finer points of the x86
> > instruction set and how opcodes map to CPU flags and architecture
> > extensions.
> 
> Sadly not. I only remember that the Geode LX is 1 instruction short of
> a standard 686.

There has been discussion on the debian-release mailing list about the
extent to which 32-bit x86 can be supported in the future, which might
be relevant to your interests. It seems unlikely that i386 will survive
as a full port providing a bootable system for the benefit of older
hardware like the Geode unless there are enough volunteers for an i386
porting team with knowledge of its quirks (i387 FPU, etc.) to be able
to fix things like this.

    smcv

Message #82 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org, 976374@bugs.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Thu, 10 Dec 2020 21:43:18 +0200

to 10. jouluk. 2020 klo 21.21 Simon McVittie (smcv@debian.org) kirjoitti:
> On Thu, 10 Dec 2020 at 20:02:42 +0200, Martin-Éric Racine wrote:
> > Core was generated by `/usr/bin/gnome-shell'.
> > Program terminated with signal SIGILL, Illegal instruction.
> > #0  0xb5500d08 in compiler_builtins::int::udiv::__udivmoddi4 () from
> > /usr/lib/i386-linux-gnu/libmozjs-78.so.0
> > [Current thread is 1 (Thread 0xb15c9b40 (LWP 861))]
> > (gdb) bt full
> > #0  0xb5500d08 in compiler_builtins::int::udiv::__udivmoddi4 () from
> > /usr/lib/i386-linux-gnu/libmozjs-78.so.0
> > No symbol table info available.
>
> This looks like Rust code, presumably compiled from something like
> https://sources.debian.org/src/rustc/1.48.0+dfsg1-1/vendor/compiler_builtins/src/int/udiv.rs/
>
> Debian's rustc has a patch to reduce the i386 baseline from upstream's
> pentium4 to pentiumpro
> https://sources.debian.org/src/rustc/1.48.0+dfsg1-1/debian/patches/d-i686-baseline.patch/
> but apparently that's not sufficient for a Geode LX. i686 is in a weird
> situation where the Pentium Pro was the *first* i686 CPU, but is not a
> *baseline* i686 CPU.
>
> Presumably the root cause of #976374 in librsvg, which also contains
> Rust code, is the same.

I cannot help but wonder what platform defaults GCC uses on i386.
Given how the baseline kernel for i386 (linux-image-686 i.e. Linux for
older PCs) is configured for Geode LX, I would presume the GCC
defaults are aligned to match. AFAIK that kernel is compiled for Geode
and for Generic 686, which I presume means only using the flags that
intersect both CPU variants. It might be a good starting point for the
Rust compiler too.

> There has been discussion on the debian-release mailing list about the
> extent to which 32-bit x86 can be supported in the future, which might
> be relevant to your interests. It seems unlikely that i386 will survive
> as a full port providing a bootable system for the benefit of older
> hardware like the Geode unless there are enough volunteers for an i386
> porting team with knowledge of its quirks (i387 FPU, etc.) to be able
> to fix things like this.

I was suspecting this discussion would take place sooner or later.
From this perspective, I wouldn't be surprised if Bullseye were the
last Debian release to support non-PAE 686.

This being said, if anyone asks me, non-PAE 686 ought to be a good
enough baseline for i386. Most people who need something faster have
moved onto amd64 or some other 64-bit platform. Meanwhile, i386 very
much implies old hardware. As such, rising the baseline CPU would
needlessly kill the usefulness of the port. Doubly so considering that
Debian is pretty much the last distro that installs on a Geode LX out
of the box.

Martin-Éric

Message #87 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org, 976374@bugs.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Thu, 10 Dec 2020 23:02:44 +0200

to 10. jouluk. 2020 klo 21.43 Martin-Éric Racine
(martin-eric.racine@iki.fi) kirjoitti:
>
> to 10. jouluk. 2020 klo 21.21 Simon McVittie (smcv@debian.org) kirjoitti:
> > Debian's rustc has a patch to reduce the i386 baseline from upstream's
> > pentium4 to pentiumpro
> > https://sources.debian.org/src/rustc/1.48.0+dfsg1-1/debian/patches/d-i686-baseline.patch/
> > but apparently that's not sufficient for a Geode LX. i686 is in a weird
> > situation where the Pentium Pro was the *first* i686 CPU, but is not a
> > *baseline* i686 CPU.
> >
> > Presumably the root cause of #976374 in librsvg, which also contains
> > Rust code, is the same.
>
> I cannot help but wonder what platform defaults GCC uses on i386.
> Given how the baseline kernel for i386 (linux-image-686 i.e. Linux for
> older PCs) is configured for Geode LX, I would presume the GCC
> defaults are aligned to match. AFAIK that kernel is compiled for Geode
> and for Generic 686, which I presume means only using the flags that
> intersect both CPU variants. It might be a good starting point for the
> Rust compiler too.

I'm no expert on x86 instruction sets, but looking at GCC options
suggests that "i686" would be the same as -march=pentiumpro and
-mtune=generic, which seems to provide support for a broader range of
i686-family chips. It's just a guess (I know nothing of Rust's
building environment), but I wouldn't be surprised if the above patch
also affected -mtune, which is perhaps too tight of a CPU spec.

Again, checking what compiler and linker options are used to build
linux-image-686 might provide a clearer view of what instruction set
is a safe baseline for the i386 port.

Martin-Éric

Message #92 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Simon McVittie <smcv@debian.org>

To: Martin-Éric Racine <martin-eric.racine@iki.fi>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org, 976374@bugs.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Thu, 10 Dec 2020 21:12:43 +0000

On Thu, 10 Dec 2020 at 21:43:18 +0200, Martin-Éric Racine wrote:
> I cannot help but wonder what platform defaults GCC uses on i386.

i686 without NOPL, according to
https://www.debian.org/releases/stretch/i386/release-notes/ch-information.en.html#i386-is-now-almost-i686
(which I believe is still current).

However, rust's interpretation of what "i686" means doesn't seem to be
the same as gcc's:
https://internals.rust-lang.org/t/is-pentium4-still-an-appropriate-base-cpu-for-i686/7353

> This being said, if anyone asks me, non-PAE 686 ought to be a good
> enough baseline for i386. Most people who need something faster have
> moved onto amd64 or some other 64-bit platform. Meanwhile, i386 very
> much implies old hardware. As such, rising the baseline CPU would
> needlessly kill the usefulness of the port.

That's assuming that the only purpose of i386 is to have an OS you can
boot on 32-bit processors, but I suspect it's a lot more common this
decade to use i386 as a multiarch foreign architecture as a way to run
legacy 32-bit binaries on an x86_64 CPU, and a higher baseline would be
beneficial for that.

In particular, the current baseline doesn't assume SSE2, which
means floating-point calculations on i386 use the i387 instructions
(-mfpmath=387), which don't always give the same answers as other
architectures due to extended precision. A baseline with SSE2 would be
able to use -mfpmath=sse, which "should be considerably faster in the
majority of cases and avoid the numerical instability problems of 387
code" according to gcc documentation.

Ubuntu has dropped i386 as a bootable architecture, but continues to
compile a subset for multiarch use. I'm sure Debian will eventually do the
same - I don't know whether it'll be for bullseye or for a future release.

The way to influence these decisions, if you want to, would be to
volunteer to maintain the i386 port and be one of the people responsible
for fixing bugs like this.

    smcv

Message #97 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org, 976374@bugs.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Sat, 20 Feb 2021 16:24:03 +0200

to 10. jouluk. 2020 klo 23.12 Simon McVittie (smcv@debian.org) kirjoitti:
>
> On Thu, 10 Dec 2020 at 21:43:18 +0200, Martin-Éric Racine wrote:
> > I cannot help but wonder what platform defaults GCC uses on i386.
>
> i686 without NOPL, according to
> https://www.debian.org/releases/stretch/i386/release-notes/ch-information.en.html#i386-is-now-almost-i686
> (which I believe is still current).
>
> However, rust's interpretation of what "i686" means doesn't seem to be
> the same as gcc's:
> https://internals.rust-lang.org/t/is-pentium4-still-an-appropriate-base-cpu-for-i686/7353
>
> > This being said, if anyone asks me, non-PAE 686 ought to be a good
> > enough baseline for i386. Most people who need something faster have
> > moved onto amd64 or some other 64-bit platform. Meanwhile, i386 very
> > much implies old hardware. As such, rising the baseline CPU would
> > needlessly kill the usefulness of the port.
>
> That's assuming that the only purpose of i386 is to have an OS you can
> boot on 32-bit processors, but I suspect it's a lot more common this
> decade to use i386 as a multiarch foreign architecture as a way to run
> legacy 32-bit binaries on an x86_64 CPU, and a higher baseline would be
> beneficial for that.
>
> In particular, the current baseline doesn't assume SSE2, which
> means floating-point calculations on i386 use the i387 instructions
> (-mfpmath=387), which don't always give the same answers as other
> architectures due to extended precision. A baseline with SSE2 would be
> able to use -mfpmath=sse, which "should be considerably faster in the
> majority of cases and avoid the numerical instability problems of 387
> code" according to gcc documentation.
>
> Ubuntu has dropped i386 as a bootable architecture, but continues to
> compile a subset for multiarch use. I'm sure Debian will eventually do the
> same - I don't know whether it'll be for bullseye or for a future release.
>
> The way to influence these decisions, if you want to, would be to
> volunteer to maintain the i386 port and be one of the people responsible
> for fixing bugs like this.

Elsewhere an upstream Rust contributor recommended that Debian uses
the i585 target for Rust (and LLVM) on i386, instead of trying to
downgrade the i686 target to produce pentium-pro code. He concedes
that upstream might have made some unfortunate assumptions about what
sort of CPU features are available for the generic i686 target  e.g
SSE2, but he feels that Debian using the i586 target would make more
sense if the goal is to produce code that definitely won't
accidentally use unexpected opcodes.

Martin-Éric

Message #102 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Simon McVittie <smcv@debian.org>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org, 976374@bugs.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Thu, 13 Oct 2022 09:15:25 +0300

This upstream commit should restore rustc defaults to something that
actually works on plain old 686:

https://github.com/rust-lang/rust/pull/100969

Martin-Éric

On Sat, Feb 20, 2021 at 4:24 PM Martin-Éric Racine
<martin-eric.racine@iki.fi> wrote:
>
> to 10. jouluk. 2020 klo 23.12 Simon McVittie (smcv@debian.org) kirjoitti:
> >
> > On Thu, 10 Dec 2020 at 21:43:18 +0200, Martin-Éric Racine wrote:
> > > I cannot help but wonder what platform defaults GCC uses on i386.
> >
> > i686 without NOPL, according to
> > https://www.debian.org/releases/stretch/i386/release-notes/ch-information.en.html#i386-is-now-almost-i686
> > (which I believe is still current).
> >
> > However, rust's interpretation of what "i686" means doesn't seem to be
> > the same as gcc's:
> > https://internals.rust-lang.org/t/is-pentium4-still-an-appropriate-base-cpu-for-i686/7353
> >
> > > This being said, if anyone asks me, non-PAE 686 ought to be a good
> > > enough baseline for i386. Most people who need something faster have
> > > moved onto amd64 or some other 64-bit platform. Meanwhile, i386 very
> > > much implies old hardware. As such, rising the baseline CPU would
> > > needlessly kill the usefulness of the port.
> >
> > That's assuming that the only purpose of i386 is to have an OS you can
> > boot on 32-bit processors, but I suspect it's a lot more common this
> > decade to use i386 as a multiarch foreign architecture as a way to run
> > legacy 32-bit binaries on an x86_64 CPU, and a higher baseline would be
> > beneficial for that.
> >
> > In particular, the current baseline doesn't assume SSE2, which
> > means floating-point calculations on i386 use the i387 instructions
> > (-mfpmath=387), which don't always give the same answers as other
> > architectures due to extended precision. A baseline with SSE2 would be
> > able to use -mfpmath=sse, which "should be considerably faster in the
> > majority of cases and avoid the numerical instability problems of 387
> > code" according to gcc documentation.
> >
> > Ubuntu has dropped i386 as a bootable architecture, but continues to
> > compile a subset for multiarch use. I'm sure Debian will eventually do the
> > same - I don't know whether it'll be for bullseye or for a future release.
> >
> > The way to influence these decisions, if you want to, would be to
> > volunteer to maintain the i386 port and be one of the people responsible
> > for fixing bugs like this.
>
> Elsewhere an upstream Rust contributor recommended that Debian uses
> the i585 target for Rust (and LLVM) on i386, instead of trying to
> downgrade the i686 target to produce pentium-pro code. He concedes
> that upstream might have made some unfortunate assumptions about what
> sort of CPU features are available for the generic i686 target  e.g
> SSE2, but he feels that Debian using the i586 target would make more
> sense if the goal is to produce code that definitely won't
> accidentally use unexpected opcodes.
>
> Martin-Éric

Message #107 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Fierelier OwO <fierelier@gmail.com>

To: 973414@bugs.debian.org

Cc: rustc@packages.debian.org, 976374@bugs.debian.org, smcv@debian.org, martin-eric.racine@iki.fi

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Thu, 9 Mar 2023 16:13:36 +0100

What smcv said is probably the right idea. The Rust toolchain should
be compiled in i586, and Rust programs should be compiled to i586 as
well (I'm not sure of the specifics here).

The Rust team has been expressing issues with adapting the correct
processor definition for their i686 build, see:
https://github.com/rust-lang/compiler-team/issues/548#issuecomment-1439976916
-- It is unlikely we will see them change it now, hence the pull that
Martin linked will most likely never be merged.

I'm not sure if this is too late to change for Debian 13 by now, I'd
still love to see it happen.

Message #112 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Fierelier OwO <fierelier@gmail.com>

To: 973414@bugs.debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Thu, 9 Mar 2023 16:17:02 +0100

Debian 12* -- Sorry!

On 3/9/23, Fierelier OwO <fierelier@gmail.com> wrote:
> I'm not sure if this is too late to change for Debian 13 by now, I'd
> still love to see it happen.
>

Message #117 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Martin-Éric Racine <martin-eric.racine@iki.fi>

To: Fierelier OwO <fierelier@gmail.com>

Cc: 973414@bugs.debian.org, rustc@packages.debian.org, 976374@bugs.debian.org, smcv@debian.org

Subject: Re: Bug#973414: libmozjs-78-0: invalid opcodes when launching GDM on AMD Geode

Date: Fri, 10 Mar 2023 09:58:42 +0200

On Thu, Mar 9, 2023 at 5:13 PM Fierelier OwO <fierelier@gmail.com> wrote:
>
> What smcv said is probably the right idea. The Rust toolchain should
> be compiled in i586, and Rust programs should be compiled to i586 as
> well (I'm not sure of the specifics here).
>
> The Rust team has been expressing issues with adapting the correct
> processor definition for their i686 build, see:
> https://github.com/rust-lang/compiler-team/issues/548#issuecomment-1439976916
> -- It is unlikely we will see them change it now, hence the pull that
> Martin linked will most likely never be merged.
>
> I'm not sure if this is too late to change for Debian 13 by now, I'd
> still love to see it happen.

As far as I know, Debian already is compiling for i586 CPU on i386 as
a stopgap measure. This doesn't seem to work as expected.

It's just a hunch, but I don't think that this issue will be resolved
until Debian switches to the Rust compiler that has been merged with
GCC 13, since GCC tends to respect CPU flags across the board.

Martin-Éric

Message #122 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Fri, 24 Mar 2023 14:23:18 +0000

Package: rustc
Followup-For: Bug #973414
X-Debbugs-Cc: martin-eric.racine@iki.fi, fierelier@gmail.com, smcv@debian.org, rustc@packages.debian.org

I've been trying to track down 'NOPL' opcodes in Debian's i386 bookworm archive
and many of the affected cases appear Rust and/or LLVM-related.

The simplest way to repro a Rust example that I've found so far is to build
the src:rust-lscolors[1] package from source in an i386 bookworm environment,
and then to 'objdump -d' the resulting 'lscolors' binary to grep for 'nopl'.

During that investigation, I learned about the existence of this bug, because
it sounds similar (unsupported opcodes, Rust toolchain, i386).


On Fri, 10 Mar 2023 09:58:42 +0200, Éric wrote:
> It's just a hunch, but I don't think that this issue will be resolved
> until Debian switches to the Rust compiler that has been merged with
> GCC 13, since GCC tends to respect CPU flags across the board.

That might be true, but I'd like to hope that we might be able to do better, or
at least determine where the opcodes (division and nopl) are originating from.

Although I understand that Debian has a rustc patch[2] in place to adjust the
intended baseline, my current/next area of focus is going to be llvm-14.0.6
because as Éric notes, something doesn't seem to be functioning as intended.

In particular, one area to look at:

  https://github.com/llvm/llvm-project/blob/llvmorg-14.0.6/clang/lib/Driver/ToolChains/Arch/X86.cpp#L25-L70

(with credit to an upstream thread[3] where Fierelier is also chasing this)


[1] - https://packages.debian.org/source/bookworm/rust-lscolors

[2] - https://sources.debian.org/src/rustc/1.63.0%2Bdfsg1-2/debian/patches/d-rustc-i686-baseline.patch/

[3] - https://github.com/llvm/llvm-project/issues/61347

Message #127 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Fri, 24 Mar 2023 16:09:58 +0000

Followup-For: Bug #973414

On Fri, 24 Mar 2023 14:23:18 +0000, James wrote:
> In particular, one area to look at:
>
>   https://github.com/llvm/llvm-project/blob/llvmorg-14.0.6/clang/lib/Driver/ToolChains/Arch/X86.cpp#L25-L70

Line range correction:

  https://github.com/llvm/llvm-project/blob/llvmorg-14.0.6/clang/lib/Driver/ToolChains/Arch/X86.cpp#L25-L115

(I haven't determined whether that function is relevant yet, but it does seem
possible that anything up to and/or including the fallback at the end could be
relevant)

Message #132 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Fri, 24 Mar 2023 16:33:19 +0000

Followup-For: Bug #973414

Re: getX86TargetCPU, see also:

  * https://github.com/llvm/llvm-project/issues/15498

  * https://bugs.llvm.org/show_bug.cgi?id=2051

Message #137 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Fri, 24 Mar 2023 16:48:44 +0000

Package: rustc
Followup-For: Bug #973414

> On Fri, 24 Mar 2023 14:23:18 +0000, James wrote:
> > In particular, one area to look at:
> >
> >   https://github.com/llvm/llvm-project/blob/llvmorg-14.0.6/clang/lib/Driver/ToolChains/Arch/X86.cpp#L25-L70
>
> Line range correction:
>
>   https://github.com/llvm/llvm-project/blob/llvmorg-14.0.6/clang/lib/Driver/ToolChains/Arch/X86.cpp#L25-L115

Hmm.. we already patch the default CPU fallback (and have done since LLVM 7):

https://salsa.debian.org/pkg-llvm-team/llvm-toolchain/-/blob/14/debian/patches/clang-baseline-fix-i386.patch

Message #142 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Sat, 25 Mar 2023 16:34:16 +0000

Followup-For: Bug #973414
X-Debbugs-Cc: martin-eric.racine@iki.fi, fierelier@gmail.com, smcv@debian.org, rustc@packages.debian.org

The default behaviour of rustc to use static linking for output binaries could
be relevant to this bug.

Although I realize that the NOPL problem I'm tracking down isn't precisely the
issue originally reported here, both of them do relate to the way that the Rust
toolchain emits binaries.

On Fri, 24 Mar 2023 14:23:18 +0000, James wrote:
> The simplest way to repro a Rust example that I've found so far is to build
> the src:rust-lscolors[1] package from source in an i386 bookworm environment,
> and then to 'objdump -d' the resulting 'lscolors' binary to grep for 'nopl'.

Since then, I've found that exporting RUSTFLAGS='-C prefer-dynamic' as an
environment variable during compilation of 'src:rust-lscolors' produces a
working output binary that is free from NOPL (non-i686) instructions.

That's not a precise way to confirm that the static linking is the problem, but
it does make it much more likely, I reckon.


So, roughly speaking:

  * I think that Debian's patched LLVM is correctly matched to the Debian i386
    baseline, and behaves as expected.

  * I think that Debian's patched rustc profile for i686 GNU/Linux (as
    mentioned by smcv) is also working as expected.

  * The existing compiled Rust stdlib libraries in Debian i386 contain some
    non-i686 opcodes.

  * Compiling statically-linked Rust libraries (the default) on Debian i386
    may include non-i686 opcodes such as those from the libstd-rust-dev
    package.


If that's the case, then one question is: why do those existing i386 packages
contain those opcodes?

  $ dpkg -L libstd-rust-dev | grep builtins
  /usr/lib/rustlib/i686-unknown-linux-gnu/lib/libcompiler_builtins-8e47f837ce9e5b67.rlib

  $ objdump -d /usr/lib/rustlib/i686-unknown-linux-gnu/lib/libcompiler_builtins-8e47f837ce9e5b67.rlib | grep -wc nopl
  31


(rebuilding them, and then rebuilding their Debian i386 Rust rdepends, should
I believe fix the problem if this theory holds)

Message #147 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Mon, 27 Mar 2023 08:52:44 +0100

Package: rustc
Followup-For: Bug #973414

> If that's the case, then one question is: why do those existing i386 packages
> contain those opcodes?

When building the Rust compiler, some builtins (intrinsics?) may be compiled
from C source code.

If _those_ aren't targeted to the correct architecture, then -- because they
are part of the stdlib and potentially other binaries in the base Rust
toolchain, and statically linked into downstream Rust binaries -- that
misconfiguration can affect downstream builds.

There's some discussion about this at:

  * https://github.com/rust-lang/rust/issues/14441

  * https://github.com/rust-lang/rust/pull/31110

Today I plan to rebuild rustc on Debian i386 with RUSTC_FLAGS (different to
RUSTFLAGS) configured for i686 during the build.  That's inspired by a comment
here: https://github.com/rust-lang/rust/pull/31110#issuecomment-174327810

To implement that, I've temporarily added RUSTC_FLAGS to the list of
per-architecture buildflags accepted by dpkg-buildflags on my system.  I don't
know whether that's the best place to put that configuration, but it would
allow it to be configured on a per-architecture basis in Debian.

Message #152 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Mon, 27 Mar 2023 13:32:40 +0100

Package: rustc
Followup-For: Bug #973414

> Today I plan to rebuild rustc on Debian i386 with RUSTC_FLAGS (different to
> RUSTFLAGS) configured for i686 during the build.  That's inspired by a comment
> here: https://github.com/rust-lang/rust/pull/31110#issuecomment-174327810
> 
> To implement that, I've temporarily added RUSTC_FLAGS to the list of
> per-architecture buildflags accepted by dpkg-buildflags on my system.  I don't
> know whether that's the best place to put that configuration, but it would
> allow it to be configured on a per-architecture basis in Debian.

That didn't work; the resulting libraries continue to contain NOPL opcodes,
indicating that their C components were not compiled to the correct baseline.

The environment variable was definitely exported to the build process, so
it seems likely that I chose an incorrect environment variable name and/or
value.

Message #157 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Fierelier OwO <fierelier@gmail.com>

To: 973414@bugs.debian.org

Cc: jay@jp-hosting.net

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Wed, 29 Mar 2023 00:10:52 +0200

@jay - I hope this helps you:
- issue 1: i386, i486, i586, i686 are considered as Pentium 4 in LLVM,
which might be relevant if Rust is compiled with it:
https://github.com/llvm/llvm-project/issues/61347
- issue 2: Rust considers i686 as Pentium 4, and i586 as Pentium:
https://github.com/rust-lang/rust/issues/82435

So, what I'm thinking is:
- When building the Rust toolchain itself, if Debian uses LLVM for it,
it might be a good idea to set the CPU target explicitly to pentiumpro
in LLVM's flags (fix issue 1) (It could also be a good idea to set
this for LLVM in general if possible, if not already done so). -- Also
compile the i586 Rust toolchain, not the i686 one (fix issue 2).
- For Rust programs, I'd probably recommend setting the target for all
(?) of them to i586 (fix issue 2). One can also try to add "-C
target-cpu=pentiumpro -C target-feature=-mmx -C target-feature=-sse -C
target-feature=-sse2" to RUSTFLAGS, if the above doesn't work.

Most of this is assumptions, but I hope it brings some progress.

Best of luck and thank you.

Message #162 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Wed, 29 Mar 2023 03:13:06 +0100

Package: rustc
Followup-For: Bug #973414
X-Debbugs-Cc: fierelier@gmail.com

After a few false starts, I've built libstd-rust-dev:i386 targeting i686 in the
way I'd expected (and to clarify: the interpretation I'm using is to match
Debian's baseline and a strict-ish reading of what P6 / i686 was when
originally specified).

Editing the 'd-rustc-i686-baseline.patch'[1] to specify "i686" instead of the
current "pentiumpro" achieved the result I was looking for: absence of NOPL in
the libstd libraries included when users compile statically-linked Rust code.

I'll double-check the results soon, and go back to see how that result affects
the original bugreport here.  My hope is that it helps to verify that the
rustc target spec is where we should be looking, but further work may be
required.

Caveats / todo items:

  * Check that "i686" is a valid CPU (not architecture) target

  * Ideally, test this on relevant hardware (non-NOPL, non-SSE2?)


Apologies, Fierelier - I didn't see your update until after these results
arrived.  Your links look relevant though, yep.  I _think_ Debian has patched
both those locations; I'll check in detail within the next day or two.

[1] - https://sources.debian.org/src/rustc/1.63.0%2Bdfsg1-2/debian/patches/d-rustc-i686-baseline.patch/

Message #167 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Wed, 29 Mar 2023 18:43:45 +0100

[Message part 1 (text/plain, inline)]

Package: rustc
Followup-For: Bug #973414
X-Debbugs-Cc: debian-toolchain@lists.debian.org, martin-eric.racine@iki.fi, fierelier@gmail.com
Control: severity -1 serious
Control: tags -1 patch

A caution: there's a good chance I'm wasting everyone's time with this patch
and much of the preceding commentary about it.

However, I'd feel more comfortable raising this commotion about it and then
being talked down reasonably than I would if I didn't attempt to apply what I
feel is a fix.

Worth noting: it could still be an incorrect fix; this area is somewhat beyond
my expertise.  That's a reasonable reason to reject it too, because I don't
think anyone would want it to be applied and cause long-term problems.

Anyhow: Dear Maintainer, please find attached a patch that may address this
issue that I feel may be release critical for bookworm, per Debian's
architecture policy guidelines.

[d-rustc-i686-baseline.patch (text/plain, attachment)]

Message #176 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Fri, 31 Mar 2023 11:46:28 +0100

Package: rustc
Followup-For: Bug #973414
X-Debbugs-Cc: fierelier@gmail.com

Hi Fierelier - thanks for your previous comment, here's my reply, slightly
later than I'd hoped:

On Wed, 29 Mar 2023 00:10:52 +0200, Fierelier wrote:
> - issue 1: i386, i486, i586, i686 are considered as Pentium 4 in LLVM,
> which might be relevant if Rust is compiled with it:
> https://github.com/llvm/llvm-project/issues/61347

Yep, this is relevant.

Debian does currently use LLVM for rustc builds, yep.  Along the way I found
the rustc build README.Debian[1] that's a very useful explanatory guide.

To be slightly more specific, currently LLVM 14 is in use, and some of the
build options for it (including that version number[2]) are placed into a
config.toml[3] file that's used by the rustc build process (x.py).

Included in Debian's patches for the LLVM 14 toolchain is a patch to adjust
the Pentium 4 target you mention down to i686:

https://sources.debian.org/src/llvm-toolchain-14/1%3A14.0.6-12/debian/patches/clang-baseline-fix-i386.patch

> - issue 2: Rust considers i686 as Pentium 4, and i586 as Pentium:
> https://github.com/rust-lang/rust/issues/82435

Also relevant, yep.  Some of the comments, and the current title that I read
for that thread ("i686 target spec disagrees with downstream definitions")
discuss the core of the problem here.

There are reasons for some people to want the definition of i686 to shift
forward, perhaps towards the bulk of the processors within that family that
are in everyday use.

And there are reasons to want to keep the definition to its original, most
compatible specification (although as found on my NOPL learnings, even in the
early days of the P6 there was some divergence there).

I don't like neither wasted CPU time, nor wasted electricity, nor wasted human
time spent reading and understanding these issues, but it's difficult (for me,
at least) to see a remedy that is optimal for all of those.  Even writing an
additional paragraph about this introduces overhead for future readers.

> - When building the Rust toolchain itself, if Debian uses LLVM for it,
> it might be a good idea to set the CPU target explicitly to pentiumpro
> in LLVM's flags (fix issue 1) (It could also be a good idea to set
> this for LLVM in general if possible, if not already done so). -- Also
> compile the i586 Rust toolchain, not the i686 one (fix issue 2).

It looks like that CPU target adjustment suggestion was accepted in #908561 in
September Y2018.

For my investigation regarding the presence of NOPL -- an instruction that is
unavailable[4] on a limited number of i686 CPUs -- reducing that target further
to from "pentiumpro" to "i686" appears to resolve the issue.

That, I expect, also has the side-effect of omitting MMX instructions from the
set of valid instructions that the Debian i386 Rust toolchain may output.  So
there is at least one downside there.

My personal preference, that may or may not be aligned with Debian, is that
switching to build from the Rust i586 toolchain, while it would be a valid fix,
could cause confusion; it'd be a form of ecosystem fragmentation, and puzzling
for humans in particular to reason about.

Arguably that could be worthwhile puzzlement -- "why do Debian's Rust packages
and libraries refer to i586?", followed by learning about the situation and
knowledge spreading.  Is that better than referring to i686 as with other
toolchains within Debian, yet having some divergence from what upstream's i686
means?

In honesty, I don't know.  But I would like people who install Debian i386 on
all systems within Debian's i386 baseline to be able to use those computers and
the Debian-packaged software as intended, as completely as possible, and to me
it feels like changing the policy baseline or changing to i586 are beyond my
skills (certainly in time for bookworm), so I'm offering the best option that I
can.

Thanks again,
James

[1] - https://sources.debian.org/src/rustc/1.63.0%2Bdfsg1-2/debian/README.Debian/#L28-L32

[2] - https://sources.debian.org/src/rustc/1.63.0%2Bdfsg1-2/debian/rules/#L31

[3] - https://sources.debian.org/src/rustc/1.63.0%2Bdfsg1-2/debian/config.toml.in/#L30-L43

[4] - https://bugzilla.redhat.com/show_bug.cgi?id=579838#c32

Message #181 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Fierelier OwO <fierelier@gmail.com>

To: 973414 <973414@bugs.debian.org>

Cc: jay <jay@jp-hosting.net>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Mon, 3 Apr 2023 23:07:35 +0200

* In terms of efficiency: I believe that if people are on the hunt for
efficiency on CPUs at the top-end of x86, they shall use distros that
target Pentium 4 explicitly. There are a few of them out there, now.
Archlinux32 for example carries packages for i486, i686 and Pentium 4
where possible, and lets you enable the architectures that suit you
best.

* In terms of confusion: I think using the Rust i586 toolchain, and
building for i586 with Rust might be less confusing, because altering
the i686 definitions for rustc will make it act differently compared
to other platforms. People could compile code for i686 on Debian, get
working code, but then if someone else does the same on another
distro, the produced binary would not work on the same devices.

Although, I think changing the i686 definition down, like you did, is
the most accurate way to achieve the goal, and is what I personally
think should be mainline. If the accurate definition proves itself on
Debian, it might find itself in mainline Rust, too, which would be
awesome.

Thank you for your continued interest in this topic.

Message #186 received at 973414@bugs.debian.org (full text, mbox, reply):

From: James Addison <jay@jp-hosting.net>

To: Debian Bug Tracking System <973414@bugs.debian.org>

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Mon, 03 Apr 2023 22:35:11 +0100

Followup-For: Bug #973414
X-Debbugs-Cc: fierelier@gmail.com

On Mon, 3 Apr 2023 23:07:35 +0200, Fierelier wrote:
> * In terms of confusion: I think using the Rust i586 toolchain, and
> building for i586 with Rust might be less confusing, because altering
> the i686 definitions for rustc will make it act differently compared
> to other platforms. People could compile code for i686 on Debian, get
> working code, but then if someone else does the same on another
> distro, the produced binary would not work on the same devices.

That's a fair point, yep; matching upstream behaviour is often the simplest and
most understandable approach.

It seems the choice is between using an upstream-compatible definition, or an
inter-toolchain-compatible definition (are there any other choices?  I did
notice use of per-extension feature flags in some build configurations. they
don't appear widely standardized though, or at least that's my initial sense).

> Although, I think changing the i686 definition down, like you did, is
> the most accurate way to achieve the goal, and is what I personally
> think should be mainline. If the accurate definition proves itself on
> Debian, it might find itself in mainline Rust, too, which would be
> awesome.

Thanks - and thank you for re-stating the intent of the patch. (I confusingly
wrote "my personal preference" instead of "my personal opinion" in a previous
comment)

I wouldn't want Debian's choice to be intended as a way to influence Rust's
upstream choices.  It'd be more a case of "who is comfortable using a divergent
definition of "i686" for longer before their respective communities call them
out on it" (in other words: each project can take their own path, and each
project can decide how to proceed, as they usually would).

Message #191 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Paul Gevers <elbrus@debian.org>

To: 973414@bugs.debian.org

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Wed, 26 Apr 2023 22:09:53 +0200

[Message part 1 (text/plain, inline)]

Control: severity -1 important

Hi,

We discussed this during the Release Team IRC meeting [1], and we 
decided that it's too late in the release cycle to fix this. We'll 
accept the de-facto baseline bump for bookworm and document the fact in 
the Release Notes.

Having said that, without having discussed it, I don't think we object 
having support for the previous baseline in trixie, if we still release 
i386 with it.

Paul

[OpenPGP_signature (application/pgp-signature, attachment)]

Message #198 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Paul Gevers <elbrus@debian.org>

To: 973414@bugs.debian.org

Subject: Re: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Wed, 26 Apr 2023 22:11:11 +0200

[Message part 1 (text/plain, inline)]

Oops, link missing.

On 26-04-2023 22:09, Paul Gevers wrote:
> We discussed this during the Release Team IRC meeting [1]

[1] 
http://meetbot.debian.net/debian-release/2023/debian-release.2023-04-26-18.59.html

Paul

[OpenPGP_signature (application/pgp-signature, attachment)]

Message #203 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Samuel Thibault <sthibault@debian.org>

To: 973414@bugs.debian.org

Subject: Re: Bug#973414: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Tue, 4 Feb 2025 12:09:57 +0100

Hello,

Paul Gevers, le mer. 26 avril 2023 22:09:53 +0200, a ecrit:
> We discussed this during the Release Team IRC meeting [1]
> http://meetbot.debian.net/debian-release/2023/debian-release.2023-04-26-18.59.html
> , and we decided that it's too late in the release cycle to fix
> this. We'll accept the de-facto baseline bump for bookworm and
> document the fact in the Release Notes.
> 
> Having said that, without having discussed it, I don't think we object
> having support for the previous baseline in trixie, if we still release i386
> with it.

AIUI in the end rustc is patching i686 to make it pentiumpro-based, so
not actually a baseline bump?

This patching is being questioned:
https://github.com/rust-lang/rust/issues/136495#issuecomment-2632546175
(without apparently realizing that the question should be asked here,
not there...)

The issue is that without SSE2 it's apparently very hard for rustc to
produce correct floating-point results, so the Debian patching makes
some software not really behave correctly, which then gets reported
to rust upstream, which thus get annoyed because "works for us" (with
SSE2).

I'm wondering: since Debian 13 won't provide an i386 kernel and thus
only support i386 chroots on otherwise-x86_64 machines, perhaps
we can indeed really bump the i386 ABI to SSE2 and thus avoid the
floating-point issues and avoid departing from upstream?

Samuel

Message #208 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Samuel Thibault <sthibault@debian.org>

To: 973414@bugs.debian.org

Subject: Re: Bug#973414: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Tue, 4 Feb 2025 23:10:15 +0100

Samuel Thibault, le mar. 04 févr. 2025 12:09:57 +0100, a ecrit:
> Paul Gevers, le mer. 26 avril 2023 22:09:53 +0200, a ecrit:
> > We discussed this during the Release Team IRC meeting [1]
> > http://meetbot.debian.net/debian-release/2023/debian-release.2023-04-26-18.59.html
> > , and we decided that it's too late in the release cycle to fix
> > this. We'll accept the de-facto baseline bump for bookworm and
> > document the fact in the Release Notes.
> > 
> > Having said that, without having discussed it, I don't think we object
> > having support for the previous baseline in trixie, if we still release i386
> > with it.
> 
> AIUI in the end rustc is patching i686 to make it pentiumpro-based, so
> not actually a baseline bump?

And it was discussed on
https://lists.debian.org/debian-devel/2024/11/msg00328.html
and again today on IRC. Consensus seems to be to raise to SSE2, which
would fix the x87 unsoundness.

Samuel

> This patching is being questioned:
> https://github.com/rust-lang/rust/issues/136495#issuecomment-2632546175
> (without apparently realizing that the question should be asked here,
> not there...)
> 
> The issue is that without SSE2 it's apparently very hard for rustc to
> produce correct floating-point results, so the Debian patching makes
> some software not really behave correctly, which then gets reported
> to rust upstream, which thus get annoyed because "works for us" (with
> SSE2).
> 
> I'm wondering: since Debian 13 won't provide an i386 kernel and thus
> only support i386 chroots on otherwise-x86_64 machines, perhaps
> we can indeed really bump the i386 ABI to SSE2 and thus avoid the
> floating-point issues and avoid departing from upstream?
> 
> Samuel

Message #213 received at 973414@bugs.debian.org (full text, mbox, reply):

From: Fabian Grünbichler <debian@fabian.gruenbichler.email>

To: 973414@bugs.debian.org

Cc: Samuel Thibault <sthibault@debian.org>

Subject: Re: Bug#973414: rustc: produces non-baseline opcodes for compiler_builtins::int::udiv::__udivmoddi4 on i386

Date: Thu, 13 Feb 2025 08:30:23 +0100

On Tue, 4 Feb 2025 23:10:15 +0100 Samuel Thibault <sthibault@debian.org> wrote:
> Samuel Thibault, le mar. 04 févr. 2025 12:09:57 +0100, a ecrit:
> > Paul Gevers, le mer. 26 avril 2023 22:09:53 +0200, a ecrit:
> > > We discussed this during the Release Team IRC meeting [1]
> > > http://meetbot.debian.net/debian-release/2023/debian-release.2023-04-26-18.59.html
> > > , and we decided that it's too late in the release cycle to fix
> > > this. We'll accept the de-facto baseline bump for bookworm and
> > > document the fact in the Release Notes.
> > > 
> > > Having said that, without having discussed it, I don't think we object
> > > having support for the previous baseline in trixie, if we still release i386
> > > with it.
> > 
> > AIUI in the end rustc is patching i686 to make it pentiumpro-based, so
> > not actually a baseline bump?
> 
> And it was discussed on
> https://lists.debian.org/debian-devel/2024/11/msg00328.html
> and again today on IRC. Consensus seems to be to raise to SSE2, which
> would fix the x87 unsoundness.

I filed a separate bug for this now (well, several actually):

#1095862 (RC bug on rustc, cloned to llvm-18 and llvm-19 as those are
slated for Trixie)
#1095863 (release.debian.org bug for raising the baseline)

This bug here was about an issue with the same patch that accidentally
didn't lower the baseline enough back before the bookworm release, which
was not fixed on the rustc side but rather "solved" with a baseline
raise ;)

Send a report that this bug log contains spam.