Debian Bug report logs - #983818
linux-image-5.10.0-3-arm64: often fails to bring up ethernet / dwmac_rk module on rockpro64

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Forest <forestix@sonic.net>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: linux-image-5.10.0-3-arm64: often fails to bring up eth0 / dwmac_rk module

Date: Mon, 01 Mar 2021 15:33:27 -0800

Package: src:linux
Version: 5.10.13-1
Severity: critical
Justification: breaks unrelated software

Dear Maintainer,

When booting recent kernels on a RockPro64 board (rk3399), eth0 often fails
to come up, leaving this headless box practically unusable without serial
console intervention. Logging in on the console and using rmmod/modprobe to
reload dwmac_rk revives the network interface and allows normal operation
until the next reboot.

Logs are included below, but the last relevant dmesg errors seem to be:
rk_gmac-dwmac fe300000.ethernet eth0: stmmac_hw_setup: DMA engine initialization failed
rk_gmac-dwmac fe300000.ethernet eth0: stmmac_open: Hw setup failed

The problem doesn't occur on every boot. I haven't determined what conditions
make it more likely. It's possible that scheduling an fsck makes it happen
less frequently (perhaps there's a timing issue?) but it's hard to say; it
still happens fairly often even when I fsck on every boot.

The problem appeared some time in the past two or three months. I keep up
with unstable kernel updates, so maybe a semi-recent change in the unstable
kernel caused it? Or maybe I was just lucky until recently.

Curiously, when dropbear ssh launches from initramfs, it never has trouble
with eth0. The problem doesn't show up until after dropbear has run and I
have unlocked my root filesystem over ssh and boot continues. I am using an
initramfs-tools tweak to make dropbear work around #968519, so I suppose that
bug and the present problem could be related but merely avoided by dropbear.
However, #968519 was present long before the present problem appeared.

Here's someone else experiencing this problem:
https://forum.pine64.org/showthread.php?tid=9351&pid=87304#pid87304


-- Package-specific info:
** Version:
Linux version 5.10.0-3-arm64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.1) #1 SMP Debian 5.10.13-1 (2021-02-06)

** Command line:
root=/dev/mapper/sda1_crypt console=ttyS2,115200n8

** Tainted: U (64)
 * taint requested by userspace application

** Kernel log:
[   67.701683] systemd[1]: Created slice system-serial\x2dgetty.slice.
[   67.725820] systemd[1]: Listening on Journal Socket.
[   67.746229] systemd[1]: Starting Create list of required static device nodes for the current kernel...
[   67.778120] systemd[1]: Mounting Huge Pages File System...
[   68.205792] EXT4-fs (dm-0): re-mounted. Opts: errors=remount-ro
[   68.628963] Adding 4194300k swap on /swapfile.  Priority:-2 extents:3 across:4464636k FS
[   68.695781] systemd-journald[1431]: Received request to flush runtime journal from PID 1
[   69.272221] cpu cpu0: EM: hertz/watts ratio non-monotonically decreasing: em_perf_state 1 >= em_perf_state0
[   69.272543] cpu cpu0: EM: created perf domain
[   69.283388] cpu cpu4: EM: created perf domain
[   69.325716] dma-pl330 ff6d0000.dma-controller: Loaded driver for PL330 DMAC-241330
[   69.334210] dma-pl330 ff6d0000.dma-controller: 	DBUFF-32x8bytes Num_Chans-6 Num_Peri-12 Num_Events-12
[   69.341198] dma-pl330 ff6e0000.dma-controller: Loaded driver for PL330 DMAC-241330
[   69.353042] dma-pl330 ff6e0000.dma-controller: 	DBUFF-128x8bytes Num_Chans-8 Num_Peri-20 Num_Events-16
[   69.466554] alg: No test for fips(ansi_cprng) (fips_ansi_cprng)
[   69.498011] dw_wdt ff848000.watchdog: No valid TOPs array specified
[   69.523015] vdd_log: supplied by regulator-dummy
[   69.529289] panfrost ff9a0000.gpu: clock rate = 500000000
[   69.659299] panfrost ff9a0000.gpu: mali-t860 id 0x860 major 0x2 minor 0x0 status 0x0
[   69.668037] panfrost ff9a0000.gpu: features: 00000000,100e77bf, issues: 00000000,24040400
[   69.669770] spi-nor spi0.0: gd25q128 (16384 Kbytes)
[   69.677219] panfrost ff9a0000.gpu: Features: L2:0x07120206 Shader:0x00000000 Tiler:0x00000809 Mem:0x1 MMU:0x00002830 AS:0xff JS:0x7
[   69.695939] panfrost ff9a0000.gpu: shader_present=0xf l2_present=0x1
[   69.704295] [drm] Initialized panfrost 1.1.0 20180908 for ff9a0000.gpu on minor 0
[   69.742766] Bluetooth: Core ver 2.22
[   69.747132] NET: Registered protocol family 31
[   69.752235] Bluetooth: HCI device and connection manager initialized
[   69.759758] Bluetooth: HCI socket layer initialized
[   69.765366] Bluetooth: L2CAP socket layer initialized
[   69.771094] Bluetooth: SCO socket layer initialized
[   69.819974] sd 3:0:0:0: Attached scsi generic sg0 type 0
[   69.836079] rockchip-vop ff8f0000.vop: Adding to iommu group 2
[   69.845013] rockchip-vop ff900000.vop: Adding to iommu group 3
[   69.862295] rockchip-drm display-subsystem: bound ff8f0000.vop (ops vop_component_ops [rockchipdrm])
[   69.872663] [drm] unsupported AFBC format[3231564e]
[   69.881216] rockchip-drm display-subsystem: bound ff900000.vop (ops vop_component_ops [rockchipdrm])
[   69.891809] dwhdmi-rockchip ff940000.hdmi: Detected HDMI TX controller v2.11a with HDCP (DWC HDMI 2.0 TX PHY)
[   69.903887] rockchip-drm display-subsystem: bound ff940000.hdmi (ops dw_hdmi_rockchip_ops [rockchipdrm])
[   69.915031] rockchip-drm display-subsystem: [drm] Cannot find any crtc or sizes
[   69.924280] [drm] Initialized rockchip 1.0.0 20140818 for display-subsystem on minor 1
[   69.934777] Bluetooth: HCI UART driver ver 2.3
[   69.939788] Bluetooth: HCI UART protocol H4 registered
[   69.946837] Bluetooth: HCI UART protocol LL registered
[   69.952623] Bluetooth: HCI UART protocol ATH3K registered
[   69.958875] Bluetooth: HCI UART protocol Three-wire (H5) registered
[   69.966245] Bluetooth: HCI UART protocol Intel registered
[   69.973423] Bluetooth: HCI UART protocol Broadcom registered
[   69.980425] Bluetooth: HCI UART protocol QCA registered
[   69.986306] Bluetooth: HCI UART protocol AG6XX registered
[   69.992642] Bluetooth: HCI UART protocol Marvell registered
[   70.039649] asoc-simple-card hdmi-sound: ASoC: no DMI vendor name!
[   70.093137] dw-apb-uart ff180000.serial: failed to request DMA
[   70.500346] EXT4-fs (mmcblk1p2): mounting ext2 file system using the ext4 subsystem
[   70.522157] EXT4-fs (mmcblk1p2): mounted filesystem without journal. Opts: (null)
[   70.945054] rockchip-drm display-subsystem: [drm] Cannot find any crtc or sizes
[   71.229145] EXT4-fs (dm-1): mounted filesystem with ordered data mode. Opts: (null)
[   71.604044] audit: type=1400 audit(1614638283.976:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=1581 comm="apparmor_parser"
[   71.621477] audit: type=1400 audit(1614638283.976:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=1581 comm="apparmor_parser"
[   71.639450] audit: type=1400 audit(1614638283.984:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/bin/man" pid=1579 comm="apparmor_parser"
[   71.656523] audit: type=1400 audit(1614638283.984:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_filter" pid=1579 comm="apparmor_parser"
[   71.673314] audit: type=1400 audit(1614638283.984:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="man_groff" pid=1579 comm="apparmor_parser"
[   71.701517] audit: type=1400 audit(1614638284.076:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/sbin/tcpdump" pid=1580 comm="apparmor_parser"
[   72.041055] rk_gmac-dwmac fe300000.ethernet eth0: PHY [stmmac-0:00] driver [RTL8211F Gigabit Ethernet] (irq=POLL)
[   72.224979] Bluetooth: hci0: command 0x0c03 tx timeout
[   72.260897] rk_gmac-dwmac fe300000.ethernet: Failed to reset the dma
[   72.268073] rk_gmac-dwmac fe300000.ethernet eth0: stmmac_hw_setup: DMA engine initialization failed
[   72.278249] rk_gmac-dwmac fe300000.ethernet eth0: stmmac_open: Hw setup failed
[   80.352983] Bluetooth: hci0: BCM: Reset failed (-110)
[  133.890343] Setting dangerous option permit_mounts_in_userns - tainting kernel
[  133.900161] overlayfs: overlayfs: Allowing overlay mounts in user namespaces bears security risks
[  157.689418] rk_gmac-dwmac fe300000.ethernet eth0: stmmac_dvr_remove: removing driver
[  161.574915] rk_gmac-dwmac fe300000.ethernet: IRQ eth_wake_irq not found
[  161.582409] rk_gmac-dwmac fe300000.ethernet: IRQ eth_lpi not found
[  161.589555] rk_gmac-dwmac fe300000.ethernet: PTP uses main clock
[  161.596764] rk_gmac-dwmac fe300000.ethernet: clock input or output? (input).
[  161.604738] rk_gmac-dwmac fe300000.ethernet: TX delay(0x28).
[  161.611129] rk_gmac-dwmac fe300000.ethernet: RX delay(0x11).
[  161.617556] rk_gmac-dwmac fe300000.ethernet: integrated PHY? (no).
[  161.624596] rk_gmac-dwmac fe300000.ethernet: cannot get clock clk_mac_speed
[  161.632416] rk_gmac-dwmac fe300000.ethernet: clock input from PHY
[  161.644280] rk_gmac-dwmac fe300000.ethernet: init for RGMII
[  161.651986] rk_gmac-dwmac fe300000.ethernet: User ID: 0x10, Synopsys ID: 0x35
[  161.659983] rk_gmac-dwmac fe300000.ethernet: 	DWMAC1000
[  161.665847] rk_gmac-dwmac fe300000.ethernet: DMA HW capability register supported
[  161.674242] rk_gmac-dwmac fe300000.ethernet: RX Checksum Offload Engine supported
[  161.682623] rk_gmac-dwmac fe300000.ethernet: COE Type 2
[  161.688479] rk_gmac-dwmac fe300000.ethernet: TX Checksum insertion supported
[  161.696375] rk_gmac-dwmac fe300000.ethernet: Wake-Up On Lan supported
[  161.703924] rk_gmac-dwmac fe300000.ethernet: Normal descriptors
[  161.710588] rk_gmac-dwmac fe300000.ethernet: Ring mode enabled
[  161.717138] rk_gmac-dwmac fe300000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[  161.830517] libphy: stmmac: probed
[  161.834398] RTL8211F Gigabit Ethernet stmmac-0:00: attached PHY driver [RTL8211F Gigabit Ethernet] (mii_bus:phy_addr=stmmac-0:00, irq=POLL)
[  161.848648] RTL8211F Gigabit Ethernet stmmac-0:01: attached PHY driver [RTL8211F Gigabit Ethernet] (mii_bus:phy_addr=stmmac-0:01, irq=POLL)
[  162.048288] rk_gmac-dwmac fe300000.ethernet eth0: PHY [stmmac-0:00] driver [RTL8211F Gigabit Ethernet] (irq=POLL)
[  162.068869] rk_gmac-dwmac fe300000.ethernet eth0: No Safety Features support found
[  162.077369] rk_gmac-dwmac fe300000.ethernet eth0: PTP not supported by HW
[  162.085535] rk_gmac-dwmac fe300000.ethernet eth0: configuring for phy/rgmii link mode
[  165.153892] rk_gmac-dwmac fe300000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[  165.163627] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

** Model information
Device Tree model: Pine64 RockPro64 v2.0

** Loaded modules:
dwmac_rk
overlay
snd_soc_hdmi_codec
dw_hdmi_i2s_audio
hci_uart
btqca
btrtl
btbcm
btintel
rockchipdrm
sg
dw_mipi_dsi
ofpart
snd_soc_rockchip_i2s
snd_soc_rockchip_pcm
snd_soc_simple_card
snd_soc_simple_card_utils
bluetooth
dw_hdmi
cec
snd_soc_core
spi_nor
governor_simpleondemand
analogix_dp
mtd
evdev
jitterentropy_rng
io_domain
snd_pcm_dmaengine
panfrost
pwm_rockchip
snd_pcm
dw_wdt
leds_gpio
drbg
snd_timer
gpu_sched
drm_kms_helper
ansi_cprng
pwm_fan
snd
ecdh_generic
rfkill
soundcore
rockchip_thermal
ecc
drm
nvmem_rockchip_efuse
pl330
cpufreq_dt
ip_tables
x_tables
autofs4
ext4
crc16
mbcache
jbd2
crc32c_generic
algif_skcipher
af_alg
sd_mod
t10_pi
crc_t10dif
xhci_plat_hcd
crct10dif_generic
xhci_hcd
realtek
dm_crypt
dm_mod
ahci
libahci
libata
scsi_mod
rtc_rk808
clk_rk808
rk808_regulator
dwc3
udc_core
roles
aes_ce_blk
crypto_simd
ulpi
cryptd
aes_ce_cipher
fan53555
rk808
crct10dif_ce
crct10dif_common
ghash_ce
stmmac_platform
gf128mul
libaes
stmmac
sha2_ce
pcs_xpcs
sha256_arm64
phylink
sha1_ce
pwm_regulator
of_mdio
dwc3_of_simple
fixed
fixed_phy
ohci_platform
libphy
ohci_hcd
gpio_keys
ehci_platform
ehci_hcd
ptp
sdhci_of_arasan
usbcore
sdhci_pltfm
phy_rockchip_emmc
cqhci
phy_rockchip_pcie
phy_rockchip_typec
phy_rockchip_inno_usb2
sdhci
pps_core
dw_mmc_rockchip
dw_mmc_pltfm
spi_rockchip
i2c_rk3x
usb_common
dw_mmc

** Network interface configuration:
*** /etc/network/interfaces:
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
# "auto eth0" added to make systemd network-online.target work -forest
auto eth0
allow-hotplug eth0
iface eth0 inet dhcp

** Network status:
*** IP interfaces and addresses:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether c2:cd:eb:ff:70:bf brd ff:ff:ff:ff:ff:ff
    inet 10.1.1.112/24 brd 10.1.1.255 scope global dynamic eth0
       valid_lft 85817sec preferred_lft 85817sec
    inet6 fe80::c0cd:ebff:feff:70bf/64 scope link 
       valid_lft forever preferred_lft forever

*** Device statistics:
Inter-|   Receive                                                |  Transmit
 face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
    lo:     240       4    0    0    0     0          0         0      240       4    0    0    0     0       0          0
  eth0:   96620     582    0    0    0     0          0         0    73579     558    0    0    0     0       0          0


** PCI devices:
00:00.0 PCI bridge [0604]: Fuzhou Rockchip Electronics Co., Ltd RK3399 PCI Express Root Port [1d87:0100] (prog-if 00 [Normal decode])
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 89
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00000000-00000fff
	Memory behind bridge: fa000000-fa0fffff
	Prefetchable memory behind bridge: 00000000-000fffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: <access denied>
	Kernel driver in use: pcieport

01:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller [1b4b:9235] (rev 11) (prog-if 01 [AHCI 1.0])
	Subsystem: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller [1b4b:9235]
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 90
	Region 0: I/O ports at 0000
	Region 1: I/O ports at 0000
	Region 2: I/O ports at 0000
	Region 3: I/O ports at 0000
	Region 4: I/O ports at 0000
	Region 5: Memory at fa040000 (32-bit, non-prefetchable) [size=2K]
	Expansion ROM at fa000000 [size=256K]
	Capabilities: <access denied>
	Kernel driver in use: ahci
	Kernel modules: ahci


** USB devices:
Bus 007 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 006 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 005 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 008 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub


-- System Information:
Debian Release: 10.8
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable'), (100, 'unstable')
Architecture: arm64 (aarch64)

Kernel: Linux 5.10.0-3-arm64 (SMP w/6 CPU cores)
Kernel taint flags: TAINT_USER
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages linux-image-5.10.0-3-arm64 depends on:
ii  initramfs-tools [linux-initramfs-tool]  0.133+deb10u1
ii  kmod                                    26-1
ii  linux-base                              4.6

Versions of packages linux-image-5.10.0-3-arm64 recommends:
ii  apparmor             2.13.2-10
ii  firmware-linux-free  3.4

Versions of packages linux-image-5.10.0-3-arm64 suggests:
pn  debian-kernel-handbook  <none>
pn  linux-doc-5.10          <none>

Versions of packages linux-image-5.10.0-3-arm64 is related to:
pn  firmware-amd-graphics     <none>
pn  firmware-atheros          <none>
pn  firmware-bnx2             <none>
pn  firmware-bnx2x            <none>
pn  firmware-brcm80211        <none>
pn  firmware-cavium           <none>
pn  firmware-intel-sound      <none>
pn  firmware-intelwimax       <none>
pn  firmware-ipw2x00          <none>
pn  firmware-ivtv             <none>
pn  firmware-iwlwifi          <none>
pn  firmware-libertas         <none>
pn  firmware-linux-nonfree    <none>
pn  firmware-misc-nonfree     <none>
pn  firmware-myricom          <none>
pn  firmware-netxen           <none>
pn  firmware-qlogic           <none>
pn  firmware-realtek          <none>
pn  firmware-samsung          <none>
pn  firmware-siano            <none>
pn  firmware-ti-connectivity  <none>
pn  xen-hypervisor            <none>

-- no debconf information

Message #26 received at 983818@bugs.debian.org (full text, mbox, reply):

From: Salvatore Bonaccorso <carnil@debian.org>

To: Forest <forestix@sonic.net>, 983818@bugs.debian.org

Subject: Re: Bug#983818: linux-image-5.10.0-3-arm64: often fails to bring up eth0 / dwmac_rk module

Date: Sun, 26 Sep 2021 08:48:19 +0200

Control: tags -1 + moreinfo

On Mon, Mar 01, 2021 at 03:33:27PM -0800, Forest wrote:
> Package: src:linux
> Version: 5.10.13-1
> Severity: critical
> Justification: breaks unrelated software
> 
> Dear Maintainer,
> 
> When booting recent kernels on a RockPro64 board (rk3399), eth0 often fails
> to come up, leaving this headless box practically unusable without serial
> console intervention. Logging in on the console and using rmmod/modprobe to
> reload dwmac_rk revives the network interface and allows normal operation
> until the next reboot.
> 
> Logs are included below, but the last relevant dmesg errors seem to be:
> rk_gmac-dwmac fe300000.ethernet eth0: stmmac_hw_setup: DMA engine initialization failed
> rk_gmac-dwmac fe300000.ethernet eth0: stmmac_open: Hw setup failed
> 
> The problem doesn't occur on every boot. I haven't determined what conditions
> make it more likely. It's possible that scheduling an fsck makes it happen
> less frequently (perhaps there's a timing issue?) but it's hard to say; it
> still happens fairly often even when I fsck on every boot.
> 
> The problem appeared some time in the past two or three months. I keep up
> with unstable kernel updates, so maybe a semi-recent change in the unstable
> kernel caused it? Or maybe I was just lucky until recently.
> 
> Curiously, when dropbear ssh launches from initramfs, it never has trouble
> with eth0. The problem doesn't show up until after dropbear has run and I
> have unlocked my root filesystem over ssh and boot continues. I am using an
> initramfs-tools tweak to make dropbear work around #968519, so I suppose that
> bug and the present problem could be related but merely avoided by dropbear.
> However, #968519 was present long before the present problem appeared.
> 
> Here's someone else experiencing this problem:
> https://forum.pine64.org/showthread.php?tid=9351&pid=87304#pid87304

Could you try with the current kernel in unstable? We are at 5.14.6-2,
which had some rk3399 related changes. If you still can reproduce the
issue my best guess would be to make a report upstream, presumably
something around contacting 

Giuseppe Cavallaro <peppe.cavallaro@st.com> (supporter:STMMAC ETHERNET DRIVER)
Alexandre Torgue <alexandre.torgue@foss.st.com> (supporter:STMMAC ETHERNET DRIVER)
Jose Abreu <joabreu@synopsys.com> (supporter:STMMAC ETHERNET DRIVER)
"David S. Miller" <davem@davemloft.net> (maintainer:NETWORKING DRIVERS)
Jakub Kicinski <kuba@kernel.org> (maintainer:NETWORKING DRIVERS)
Maxime Coquelin <mcoquelin.stm32@gmail.com> (maintainer:ARM/STM32 ARCHITECTURE)
Philipp Zabel <p.zabel@pengutronix.de> (maintainer:RESET CONTROLLER FRAMEWORK)
Liam Girdwood <lgirdwood@gmail.com> (supporter:VOLTAGE AND CURRENT REGULATOR FRAMEWORK)
Mark Brown <broonie@kernel.org> (supporter:VOLTAGE AND CURRENT REGULATOR FRAMEWORK)
netdev@vger.kernel.org (open list:STMMAC ETHERNET DRIVER)
linux-stm32@st-md-mailman.stormreply.com (moderated list:ARM/STM32 ARCHITECTURE)
linux-arm-kernel@lists.infradead.org (moderated list:ARM/STM32 ARCHITECTURE)
linux-kernel@vger.kernel.org (open list)

(and keeping us downstream in the loop).

Regards,
Salvatore

Message #33 received at 983818@bugs.debian.org (full text, mbox, reply):

From: Forest <forestix@sonic.net>

To: Salvatore Bonaccorso <carnil@debian.org>, 983818@bugs.debian.org

Subject: Re: Bug#983818: linux-image-5.10.0-3-arm64: often fails to bring up eth0 / dwmac_rk module

Date: Sun, 26 Sep 2021 17:39:42 -0700

Control: tags -1 - moreinfo

>Could you try with the current kernel in unstable?
>We are at 5.14.6-2, which had some rk3399 related changes. 

Did any of those changes arrive after 5.14.0-1?  If so, I suppose I would
have to wait for a newer debian kernel to appear before I could test it.

With 5.14.0-1 (the version in unstable), the results are worse:

Dropbear no longer works.  Error message:
/scripts/init-premount/dropbear: .: line 333: can't open '/run/net-*.conf': No such file or directory

Using a serial console for LUKS unlock and then running rmmod dwmac_rk / 
modprobe dwmac_rk no longer brings up eth0.

The dmesg output has changed a bit:
$ egrep 'mac|eth0' dmesg.linux-image-5.14.0-1-arm64 
[    5.708873] rk_gmac-dwmac fe300000.ethernet: IRQ eth_wake_irq not found
[    5.709470] rk_gmac-dwmac fe300000.ethernet: IRQ eth_lpi not found
[    5.710965] rk_gmac-dwmac fe300000.ethernet: PTP uses main clock
[    5.712133] rk_gmac-dwmac fe300000.ethernet: clock input or output? (input).
[    5.713263] rk_gmac-dwmac fe300000.ethernet: TX delay(0x28).
[    5.714418] rk_gmac-dwmac fe300000.ethernet: RX delay(0x11).
[    5.716512] rk_gmac-dwmac fe300000.ethernet: integrated PHY? (no).
[    5.717492] rk_gmac-dwmac fe300000.ethernet: cannot get clock clk_mac_speed
[    5.719275] rk_gmac-dwmac fe300000.ethernet: clock input from PHY
[    5.724825] rk_gmac-dwmac fe300000.ethernet: init for RGMII
[    5.725658] rk_gmac-dwmac fe300000.ethernet: User ID: 0x10, Synopsys ID: 0x35
[    5.726328] rk_gmac-dwmac fe300000.ethernet: 	DWMAC1000
[    5.726802] rk_gmac-dwmac fe300000.ethernet: DMA HW capability register supported
[    5.727511] rk_gmac-dwmac fe300000.ethernet: RX Checksum Offload Engine supported
[    5.728183] rk_gmac-dwmac fe300000.ethernet: COE Type 2
[    5.728652] rk_gmac-dwmac fe300000.ethernet: TX Checksum insertion supported
[    5.729275] rk_gmac-dwmac fe300000.ethernet: Wake-Up On Lan supported
[    5.731134] rk_gmac-dwmac fe300000.ethernet: Normal descriptors
[    5.731674] rk_gmac-dwmac fe300000.ethernet: Ring mode enabled
[    5.732192] rk_gmac-dwmac fe300000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[    5.851291] libphy: stmmac: probed
[    5.851612] RTL8211F Gigabit Ethernet stmmac-0:00: attached PHY driver (mii_bus:phy_addr=stmmac-0:00, irq=POLL)
[    5.852504] RTL8211F Gigabit Ethernet stmmac-0:01: attached PHY driver (mii_bus:phy_addr=stmmac-0:01, irq=POLL)
[    6.639085] rk_gmac-dwmac fe300000.ethernet eth0: PHY [stmmac-0:00] driver [RTL8211F Gigabit Ethernet] (irq=POLL)
[    6.641458] rk_gmac-dwmac fe300000.ethernet eth0: Register MEM_TYPE_PAGE_POOL RxQ-0
[    6.653320] rk_gmac-dwmac fe300000.ethernet eth0: No Safety Features support found
[    6.653364] rk_gmac-dwmac fe300000.ethernet eth0: PTP not supported by HW
[    6.654183] rk_gmac-dwmac fe300000.ethernet eth0: configuring for phy/rgmii link mode
[    9.760371] rk_gmac-dwmac fe300000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[    9.760429] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[  199.685064] rk_gmac-dwmac fe300000.ethernet eth0: Link is Down
[  207.195113] rk_gmac-dwmac fe300000.ethernet eth0: PHY [stmmac-0:00] driver [RTL8211F Gigabit Ethernet] (irq=POLL)
[  207.197673] rk_gmac-dwmac fe300000.ethernet eth0: Register MEM_TYPE_PAGE_POOL RxQ-0
[  207.206976] rk_gmac-dwmac fe300000.ethernet eth0: No Safety Features support found
[  207.207681] rk_gmac-dwmac fe300000.ethernet eth0: PTP not supported by HW
[  207.208307] rk_gmac-dwmac fe300000.ethernet eth0: configuring for phy/rgmii link mode
[  210.304576] rk_gmac-dwmac fe300000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[  210.305423] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

Message #40 received at 983818@bugs.debian.org (full text, mbox, reply):

From: Diederik de Haas <didi.debian@cknow.org>

To: Forest <forestix@sonic.net>

Cc: 983818@bugs.debian.org

Subject: Re: linux-image-5.10.0-3-arm64: often fails to bring up eth0 / dwmac_rk module

Date: Sat, 30 Jul 2022 00:19:25 +0200

[Message part 1 (text/plain, inline)]

Control: tag -1 moreinfo

On Mon, 01 Mar 2021 15:33:27 -0800 Forest <forestix@sonic.net> wrote:
> Package: src:linux
> Version: 5.10.13-1
> 
> When booting recent kernels on a RockPro64 board (rk3399), eth0 often fails
> to come up, leaving this headless box practically unusable without serial
> console intervention. Logging in on the console and using rmmod/modprobe to
> reload dwmac_rk revives the network interface and allows normal operation
> until the next reboot.

Is this problem still present with a recent 5.10 or (better yet) the 5.18.14 
kernel from Unstable?

[signature.asc (application/pgp-signature, inline)]

Message #47 received at 983818@bugs.debian.org (full text, mbox, reply):

From: Forest <forestix@sonic.net>

To: Diederik de Haas <didi.debian@cknow.org>, 983818@bugs.debian.org

Subject: Re: Bug#983818: linux-image-5.10.0-3-arm64: often fails to bring up eth0 / dwmac_rk module

Date: Sat, 30 Jul 2022 16:51:08 -0700

Control: found -1 5.10.127-2
Control: notfound -1 5.18.14-1
Control: tags -1 - moreinfo

On Sat, 30 Jul 2022 00:19:25 +0200, Diederik de Haas wrote:

>Is this problem still present with a recent 5.10 or (better yet) the 5.18.14 
>kernel from Unstable?

It is still present in recent 5.10 kernels.

5.18.14-1 from unstable hasn't shown the failure in about a dozen boots.
That's encouraging.  I haven't done a bisect, but some relatively recent
commits (e.g. aec3f415) mention dwmac-rk.  Perhaps one of those fixed it?

Message #56 received at 983818@bugs.debian.org (full text, mbox, reply):

From: Diederik de Haas <didi.debian@cknow.org>

To: Forest <forestix@sonic.net>

Cc: 983818@bugs.debian.org

Subject: Re: Bug#983818: linux-image-5.10.0-3-arm64: often fails to bring up eth0 / dwmac_rk module

Date: Sun, 31 Jul 2022 12:30:42 +0200

[Message part 1 (text/plain, inline)]

On Sunday, 31 July 2022 01:51:08 CEST Forest wrote:
> >Is this problem still present with a recent 5.10 or (better yet) the
> >5.18.14 kernel from Unstable?
> 
> It is still present in recent 5.10 kernels.
> 
> 5.18.14-1 from unstable hasn't shown the failure in about a dozen boots.
> That's encouraging.  I haven't done a bisect, but some relatively recent
> commits (e.g. aec3f415) mention dwmac-rk.  Perhaps one of those fixed it?

That's certainly encouraging and the commit message makes it appear quite 
relevant indeed. 
From the partial logs you shared it appeared that your network also went down 
after (quite) some time, which is consistent with the commit message.

What's odd then is that that commit has been applied/backported to the 5.10 
kernel under commit 97653ba562b9b28e30a3fcff42531e05a434d58c which is part of 
5.10.82, so also 5.10.127-2 ...

[signature.asc (application/pgp-signature, inline)]

Message #61 received at 983818@bugs.debian.org (full text, mbox, reply):

From: Forest <forestix@sonic.net>

To: Diederik de Haas <didi.debian@cknow.org>, 983818@bugs.debian.org

Subject: Re: Bug#983818: linux-image-5.10.0-3-arm64: often fails to bring up eth0 / dwmac_rk module

Date: Sun, 31 Jul 2022 15:49:29 -0700

On Sun, 31 Jul 2022 12:30:42 +0200, Diederik de Haas wrote:

>From the partial logs you shared it appeared that your network also went down 
>after (quite) some time,

If you're referring to my 5.14.0-1 kernel log, I can't offer any insight, as
I only tried that kernel briefly, nearly a year ago.

If you mean the 5.10 kernel, let me clarify:

1. 5.10 reliably brings up eth0 early enough for dropbear sshd to work. 
2. I ssh to dropbear, enter the LUKS passphrase, and the root filesystem is
unlocked.
3. When eth0 fails, it's always shortly after that, still during system
startup.
4. Once I notice, attach a serial terminal, and reload the kernel module,
eth0 comes up and stays up. It doesn't go down again later.

I find it curious that eth0 comes up reliably and then *sometimes* goes down
shortly afterward.  I don't know if it's completely random or something
later in the startup process is triggering it.

Obviously, a delay between eth0 first coming up and when it goes down could
be partly from the time it takes me to ssh and type a LUKS passphrase.

>What's odd then is that that commit has been applied/backported to the 5.10 
>kernel under commit 97653ba562b9b28e30a3fcff42531e05a434d58c which is part of 
>5.10.82, so also 5.10.127-2 ...

Ah, I didn't notice that patch having been backported with a different
commit ID.  Thanks for mentioning it.

Message #70 received at 983818-submitter@bugs.debian.org (full text, mbox, reply):

From: Salvatore Bonaccorso <carnil@debian.org>

To: control@bugs.debian.org

Cc: 983818-submitter@bugs.debian.org

Subject: closing 983818

Date: Sat, 13 Aug 2022 14:22:46 +0200

close 983818 5.18.14-1
thanks

Send a report that this bug log contains spam.