Debian Bug report logs - #911560
u-boot: Olimex Lime2 Rev.H-L: severe packet loss w/ gigabit ethernet

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Dieter <dieter@sarina-ist-doof.de>

To: submit@bugs.debian.org

Subject: A20-OLinuXino-Lime 2 Rev. K Ethernet problem - No connection possible in installer

Date: Sun, 21 Oct 2018 21:55:28 +0200

Package: debian-installer

Hello maintainer,

i'm currently trying to install debian on a new Olimex A20 Lime 2 Rev. K.

This board uses a new ethernet-PHY, which seems to be unsupported by
current debian images. (I tried stable, as well as the installer daily
build from 2018-10-21).

The hardware-changelog says:
A20-OLinuXino-Lime2_Rev_H
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2. Ethernet PHY changed to commercial variant - KSZ9031RNXCC-TR;

I found bugreport #864562, where it is stated that one could change the
state of GPIO-Pin 17.

That however does not do anything on my board. Ethernet still does not
work.

I'll send syslog / dmesg-output when i get screen to finally save a log.

Thank you and best regards,

Dieter

Message #10 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dieter <dieter@sarina-ist-doof.de>

To: 911560@bugs.debian.org

Subject: still no syslog available

Date: Mon, 22 Oct 2018 21:21:15 +0200

Hello,

i have not had a lot of time to look into this today.

However, i found this line in the syslog:
Jan  1 00:00:27 kernel: [    7.327358] Micrel KSZ9031 Gigabit PHY
stmmac-0:01: attached PHY driver [Micrel KSZ9031 Gigabit PHY]
(mii_bus:phy_addr=stmmac-0:01, irq=POLL)


And the kernel seems to detect the ethernet also:

Jan  1 00:00:27 kernel: [    7.002010] sun7i-dwmac 1c50000.ethernet: PTP
uses main clock
Jan  1 00:00:27 kernel: [    7.083443] sun7i-dwmac 1c50000.ethernet: no
reset control found
Jan  1 00:00:27 kernel: [    7.112287] sun7i-dwmac 1c50000.ethernet: no
regulator found
Jan  1 00:00:27 kernel: [    7.135027] sun7i-dwmac 1c50000.ethernet:
Version ID not available
Jan  1 00:00:27 kernel: [    7.147471] sun7i-dwmac 1c50000.ethernet:
DWMAC1000
Jan  1 00:00:27 kernel: [    7.155074] sun7i-dwmac 1c50000.ethernet: DMA
HW capability register supported
Jan  1 00:00:27 kernel: [    7.178494] sun7i-dwmac 1c50000.ethernet:
Normal descriptors
Jan  1 00:00:27 kernel: [    7.184237] sun7i-dwmac 1c50000.ethernet:
Ring mode enabled


And of course, network auto-configuration fails.
It does not work in u-boot either.

Thanks and best regards,
Dieter

Message #15 received at 911560@bugs.debian.org (full text, mbox, reply):

From: "Andreas B. Mundt" <andi@debian.org>

To: 911560@bugs.debian.org

Subject: possible fix / workaround

Date: Fri, 4 Oct 2019 17:22:37 +0200

Control: tags -1 patch


Hi all,

I tried to boot a 'Olimex A20 Lime 2 Rev. K' too, and as reported, the
ethernet PHY seems not to be initialized well and fails to work at all. 

Following the advice of the sunxi wiki [1], recompiling u-boot with
the proposed patch [2] fixed this:  Ethernet works in u-boot as well
as in the installer with the patched u-boot. 

No idea if this patch spoils the other revisions.

Best regards,

  Andi



[1] https://linux-sunxi.org/Olimex_A20-OLinuXino-Lime2#GMAC_u-boot_config

[2]
        diff --git a/configs/A20-OLinuXino-Lime2-eMMC_defconfig b/configs/A20-OLinuXino-Lime2-eMMC_defconfig
        index 7e48328290..08a7207652 100644
        --- a/configs/A20-OLinuXino-Lime2-eMMC_defconfig
        +++ b/configs/A20-OLinuXino-Lime2-eMMC_defconfig
        @@ -38,3 +38,4 @@ CONFIG_USB_EHCI_HCD=y
         CONFIG_USB_OHCI_HCD=y
         CONFIG_USB_MUSB_GADGET=y
         CONFIG_SYS_USB_EVENT_POLL_VIA_INT_QUEUE=y
        +CONFIG_GMAC_TX_DELAY=3

Message #26 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <dr@jones.dk>

To: 911560@bugs.debian.org

Subject: Re: A20-OLinuXino-Lime 2 Rev. K Ethernet problem - No connection possible in installer

Date: Mon, 07 Oct 2019 15:22:02 +0200

[Message part 1 (text/plain, inline)]

A workaround for rev. K boards not requiring recompiling code is to 
lower transfer speed to 10Mbit/s, e.g. using ethtool or using the 
Advertise option in a systemd .link file (see "man systemd.link").

(for Rev. G2 boards it is adequate to lower speed to 100Mbit/s).

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #31 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <dr@jones.dk>

To: 911560@bugs.debian.org

Subject: Re: u-boot: Very poor ethernet performance on A20 OLinuXino Lime2 Rev.G2

Date: Thu, 19 Dec 2019 02:00:53 +0100

[Message part 1 (text/plain, inline)]

[cross-posting my post from last month to related bug#927397]

Here's what I know (the "should work" entries need confirmation):

Lime2 rev.C
-----------

* Uses Realtek rev.C PHY
* Works fine in 1Gbit mode with Debian stable U-boot
* Works fine in 1Gbit mode with Debian stable kernel
* Possibly still sold as some no-eMMC no-flash options

Lime2 rev.G
-----------

* Uses Realtek rev.E PHY
* Sold as FreedomBox Edition
* Possibly still sold as some no-flash options
* Works in 100Mbit mode with Debian-stable kernel and u-boot
* Works in 1Gbit mode with Debian-stable kernel
  and custom U-boot patched to set TX_DELAY=4

Lime rev.H
----------

* Uses Micrel PHY
* Sold (at least) as flash options
* Works in 10Mbit mode with Debian stable kernel and u-boot
* Should work in 1Gbit mode with Debian-backports kernel v5.2
* Should work in 1Gbit mode with custom kernel
  patched to to set CONFIG_MICREL_PHY=y (already done with Debian kernels)
  and to include https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3aed3e2
  (applied upstream since v5.2)


With systemd-networkd, all lime2 boards work with this added as file
/etc/systemd/network/90-ethernet.link (and rev.G boards work also
with 100baset entries uncommented):

[Match]
Driver=st_gmac

[Link]
NamePolicy=keep kernel database onboard slot path
MACAddressPolicy=persistent

Advertise=10baset-half
Advertise=10baset-full
#Advertise=100baset-half
#Advertise=100baset-full
#Advertise=1000baset-half
#Advertise=1000baset-full



 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #36 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dominique Dumont <dod@debian.org>

To: debian-arm@lists.debian.org, 911560@bugs.debian.org

Subject: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Sat, 08 Feb 2020 16:07:17 +0100

On Wednesday, 5 February 2020 19:41:30 CET Vagrant Cascadian wrote:
>   A20-OLinuXino-Lime 2 Rev. K Ethernet problem - No connection possible in
> installer https://bugs.debian.org/911560

So I've applied the suggestion to build u-boot with 

CONFIG_GMAC_TX_DELAY=3

I can confirm that both LIME2 rev G2 and rev K boards have a working network 
with 1G speed with  u-boot version debian/2020.01+dfsg-1 and the tweak above.

For the record, here's the recipe to get a working network OlinuXino Lime2 
rev. K :

Setup u-boot source:
- git clone https://salsa.debian.org/debian/u-boot.git
- sudo apt build-dep u-boot

In u-boot:
- apply all debian patches: quilt push -a
- run: make CROSS_COMPILE=arm-linux-gnueabihf- A20-OLinuXino-Lime2_defconfig
- edit .config to set CONFIG_GMAC_TX_DELAY=3
- build u-boot: make CROSS_COMPILE=arm-linux-gnueabihf-


You should get a u-boot-sunxi-with-spl.bin file which is the bootloader to 
install on your SD card (see [1])

First backup your SD card
- sudo cp /dev/$sdcard backup.img

Then copy the bootloader:
- sudo dd if=u-boot-sunxi-with-spl.bin of=/dev/$sdcard bs=1024 seek=8

In both case, $sdcard is to be replaced with the block device file of your SD 
card.

Hope this helps

[1] https://linux-sunxi.org/Bootable_SD_card#Bootloader

Message #41 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dominique Dumont <dod@debian.org>, debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Sat, 08 Feb 2020 16:54:45 +0100

[Message part 1 (text/plain, inline)]

Quoting Dominique Dumont (2020-02-08 16:07:17)
> On Wednesday, 5 February 2020 19:41:30 CET Vagrant Cascadian wrote:
> >   A20-OLinuXino-Lime 2 Rev. K Ethernet problem - No connection 
> > possible in installer https://bugs.debian.org/911560
> 
> So I've applied the suggestion to build u-boot with 
> 
> CONFIG_GMAC_TX_DELAY=3
> 
> I can confirm that both LIME2 rev G2 and rev K boards have a working 
> network with 1G speed with u-boot version debian/2020.01+dfsg-1 and 
> the tweak above.

That's quite helpful. Thanks!

Even more helpful if you provide a bit more information (if possible):

I guess you mean that the negotiated ethernet mode is 1G.  What is the 
actual performance - i.e. which concrete transfer speeds is achievable, 
in each direction, for each board?

With which kind on low-level network wiring did you test the negotiated 
mode - e.g. cross-over cable to another host or gigabit-capable switch?


Would be great if you could also test with patched u-boot in stable 
Debian - so we can consider fixing this in a point release.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #46 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dominique Dumont <dod@debian.org>

To: 911560@bugs.debian.org, Jonas Smedegaard <jonas@jones.dk>

Cc: debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Sat, 08 Feb 2020 17:55:10 +0100

On samedi 8 février 2020 16:54:45 CET Jonas Smedegaard wrote:
> I guess you mean that the negotiated ethernet mode is 1G. 

yes.

> What is the
> actual performance - i.e. which concrete transfer speeds is achievable,
> in each direction, for each board?

I've seen only outgoing traffic at about 20 to 30 Mbytes/s through NFS for the 
rev. K board. 

As far as I recall, I had about 20-25 Mbytes/s with the rev. G2 board and the 
vanilla u-boot.

> With which kind on low-level network wiring did you test the negotiated
> mode - e.g. cross-over cable to another host or gigabit-capable switch?

standard cable to a gigabit-capable switch.

> Would be great if you could also test with patched u-boot in stable
> Debian - so we can consider fixing this in a point release.

I can easily test with a patched u-boot from stable with a Debian/unstable. 
Would that be helpful ?

All the best

Dod

Message #51 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dominique Dumont <dod@debian.org>

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Sat, 08 Feb 2020 19:33:38 +0100

[Message part 1 (text/plain, inline)]

Quoting Dominique Dumont (2020-02-08 17:55:10)
> On samedi 8 février 2020 16:54:45 CET Jonas Smedegaard wrote:
> > I guess you mean that the negotiated ethernet mode is 1G. 
> 
> yes.
> 
> > What is the actual performance - i.e. which concrete transfer speeds 
> > is achievable, in each direction, for each board?
> 
> I've seen only outgoing traffic at about 20 to 30 Mbytes/s through NFS 
> for the rev. K board.
> 
> As far as I recall, I had about 20-25 Mbytes/s with the rev. G2 board 
> and the vanilla u-boot.
> 
> > With which kind on low-level network wiring did you test the 
> > negotiated mode - e.g. cross-over cable to another host or 
> > gigabit-capable switch?
> 
> standard cable to a gigabit-capable switch.

Thanks!


> > Would be great if you could also test with patched u-boot in stable 
> > Debian - so we can consider fixing this in a point release.
> 
> I can easily test with a patched u-boot from stable with a 
> Debian/unstable. Would that be helpful ?

Yes, that would be helpful.  Thanks!


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #56 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dominique Dumont <dod@debian.org>

To: 911560@bugs.debian.org, Jonas Smedegaard <jonas@jones.dk>

Cc: debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Sun, 09 Feb 2020 17:07:12 +0100

On Saturday, 8 February 2020 16:54:45 CET Jonas Smedegaard wrote:
> Would be great if you could also test with patched u-boot in stable
> Debian - so we can consider fixing this in a point release.

I've tested together:
- Debian's u-boot debian/2019.01+dfsg-7 with CONFIG_GMAC_TX_DELAY=3 tweak
- Debian's stable partition (dated 2019-11)

with Lime2 Rev G2 and rev K cards.

In both case the installer starts fine, sets network correctly (with dhcp) and 
starts downloading packages.

Because I have a crappy connection and no use for a Lime2 Debian/stable 
installation, I've stopped the tests at that point.

All the best

Message #61 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dominique Dumont <dod@debian.org>

Cc: debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Sun, 09 Feb 2020 17:17:57 +0100

[Message part 1 (text/plain, inline)]

Quoting Dominique Dumont (2020-02-09 17:07:12)
> On Saturday, 8 February 2020 16:54:45 CET Jonas Smedegaard wrote:
> > Would be great if you could also test with patched u-boot in stable
> > Debian - so we can consider fixing this in a point release.
> 
> I've tested together:
> - Debian's u-boot debian/2019.01+dfsg-7 with CONFIG_GMAC_TX_DELAY=3 tweak
> - Debian's stable partition (dated 2019-11)
> 
> with Lime2 Rev G2 and rev K cards.
> 
> In both case the installer starts fine, sets network correctly (with dhcp) and 
> starts downloading packages.
> 
> Because I have a crappy connection and no use for a Lime2 Debian/stable 
> installation, I've stopped the tests at that point.

Great!  Very helpful.

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #66 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Vagrant Cascadian <vagrant@debian.org>

To: dod@debian.org, 911560@bugs.debian.org, Jonas Smedegaard <jonas@jones.dk>

Cc: debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Sun, 09 Feb 2020 18:39:39 -0800

[Message part 1 (text/plain, inline)]

On 2020-02-09, Dominique Dumont wrote:
> On Saturday, 8 February 2020 16:54:45 CET Jonas Smedegaard wrote:
>> Would be great if you could also test with patched u-boot in stable
>> Debian - so we can consider fixing this in a point release.
>
> I've tested together:
> - Debian's u-boot debian/2019.01+dfsg-7 with CONFIG_GMAC_TX_DELAY=3 tweak
> - Debian's stable partition (dated 2019-11)

Thanks for debugging the issue!

Please submit a patch to upstream fixing this in the appropriate files
in configs/*; I'd guess configs/A20-OLinuXino-Lime2-eMMC_defconfig
and/or configs/A20-OLinuXino-Lime2_defconfig.

Then I'll feel happier about backporting them to u-boot in stable and
eventually can drop the patch...


live well,
  vagrant

[signature.asc (application/pgp-signature, inline)]

Message #71 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dominique Dumont <dod@debian.org>

To: Vagrant Cascadian <vagrant@debian.org>

Cc: 911560@bugs.debian.org, Jonas Smedegaard <jonas@jones.dk>, debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Tue, 11 Feb 2020 12:13:23 +0100

On Monday, 10 February 2020 03:39:39 CET Vagrant Cascadian wrote:
> Please submit a patch to upstream fixing this in the appropriate files
> in configs/*; I'd guess configs/A20-OLinuXino-Lime2-eMMC_defconfig
> and/or configs/A20-OLinuXino-Lime2_defconfig.

It looks like a similar patches was submitted 4 years ago and the tests of 
this patch did not go as well as I experienced:

https://lists.denx.de/pipermail/u-boot/2016-March/248498.html

I'll search in u-boot archive for other threads

All the best

Message #76 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: Dominique Dumont <dod@debian.org>, Vagrant Cascadian <vagrant@debian.org>

Cc: 911560@bugs.debian.org, debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Tue, 11 Feb 2020 12:23:06 +0100

[Message part 1 (text/plain, inline)]

Quoting Dominique Dumont (2020-02-11 12:13:23)
> On Monday, 10 February 2020 03:39:39 CET Vagrant Cascadian wrote:
> > Please submit a patch to upstream fixing this in the appropriate 
> > files in configs/*; I'd guess 
> > configs/A20-OLinuXino-Lime2-eMMC_defconfig and/or 
> > configs/A20-OLinuXino-Lime2_defconfig.
> 
> It looks like a similar patches was submitted 4 years ago and the 
> tests of this patch did not go as well as I experienced:
> 
> https://lists.denx.de/pipermail/u-boot/2016-March/248498.html

Seems to me that patch uses a different value, and does not speak about 
testing against all variants, or even which variant it intents to fix.

...speaking of "all variants", someone (you, Dominique?) should test 
also against the "original" LIME2 rev. C or older, to ensure there is no 
regression there!


> I'll search in u-boot archive for other threads

Thanks.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #81 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dominique Dumont <dod@debian.org>

To: 911560@bugs.debian.org, Jonas Smedegaard <jonas@jones.dk>

Cc: Vagrant Cascadian <vagrant@debian.org>, debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Tue, 11 Feb 2020 13:11:40 +0100

On Tuesday, 11 February 2020 12:23:06 CET Jonas Smedegaard wrote:
> someone (you, Dominique?) should test
> also against the "original" LIME2 rev. C or older, to ensure there is no
> regression there!

I don't have any other Lime2 board.

All the best

Message #86 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dominique Dumont <dod@debian.org>

To: 911560@bugs.debian.org

Cc: Vagrant Cascadian <vagrant@debian.org>, Jonas Smedegaard <jonas@jones.dk>, debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Tue, 11 Feb 2020 14:16:22 +0100

On Tuesday, 11 February 2020 12:13:23 CET Dominique Dumont wrote:
> I'll search in u-boot archive for other threads

This is promising:

https://lists.denx.de/pipermail/u-boot/2016-March/249735.html

Message #91 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dominique Dumont <dod@debian.org>

Cc: Vagrant Cascadian <vagrant@debian.org>, debian-arm@lists.debian.org

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Tue, 11 Feb 2020 16:12:42 +0100

[Message part 1 (text/plain, inline)]

Quoting Dominique Dumont (2020-02-11 14:16:22)
> On Tuesday, 11 February 2020 12:13:23 CET Dominique Dumont wrote:
> > I'll search in u-boot archive for other threads
> 
> This is promising:
> 
> https://lists.denx.de/pipermail/u-boot/2016-March/249735.html

I suspect that addresses only the issue on one of the newer LIME2 
variants, not both.  But that's just speculation - actually testing 
against all 3 variants is obviously far better ;-)


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #96 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dominique Dumont <dod@debian.org>

To: debian-arm@lists.debian.org

Cc: 911560@bugs.debian.org, Vagrant Cascadian <vagrant@debian.org>, Jonas Smedegaard <jonas@jones.dk>

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Sun, 16 Feb 2020 19:15:21 +0100

On Tuesday, 11 February 2020 14:16:22 CET Dominique Dumont wrote:
> This is promising:
> 
> https://lists.denx.de/pipermail/u-boot/2016-March/249735.html

Or not :-(

With this setup (and no GMAC_TX_DELAY), the network starts fine on rev Olimex 
LIME2 rev G.2 but not on rev K.

All the best

Message #101 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <dr@jones.dk>

To: 911560@bugs.debian.org

Subject: Re: u-boot: Very poor ethernet performance on A20 OLinuXino Lime2 Rev.G2

Date: Sun, 12 Jul 2020 02:33:14 +0200

[Message part 1 (text/plain, inline)]

Quoting Jonas Smedegaard (2019-12-19 02:00:53)
> Lime rev.H
> ----------
> 
> * Uses Micrel PHY
> * Sold (at least) as flash options
> * Works in 10Mbit mode with Debian stable kernel and u-boot
> * Should work in 1Gbit mode with Debian-backports kernel v5.2
> * Should work in 1Gbit mode with custom kernel
>   patched to to set CONFIG_MICREL_PHY=y (already done with Debian kernels)
>   and to include https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3aed3e2
>   (applied upstream since v5.2)

Recent u-boot adds an "errata 80000688A fix" for Micrel KSZ8061 - 
perhaps relevant for this issue...?

https://gitlab.denx.de/u-boot/u-boot/-/commit/baafd99


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #106 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <dr@jones.dk>

To: 911560@bugs.debian.org

Subject: Re: u-boot: Very poor ethernet performance on A20 OLinuXino Lime2 Rev.G2

Date: Sun, 12 Jul 2020 03:11:26 +0200

[Message part 1 (text/plain, inline)]

Quoting Jonas Smedegaard (2020-07-12 02:33:14)
> Quoting Jonas Smedegaard (2019-12-19 02:00:53)
> > Lime rev.H
> > ----------
> > 
> > * Uses Micrel PHY
> > * Sold (at least) as flash options
> > * Works in 10Mbit mode with Debian stable kernel and u-boot
> > * Should work in 1Gbit mode with Debian-backports kernel v5.2
> > * Should work in 1Gbit mode with custom kernel
> >   patched to to set CONFIG_MICREL_PHY=y (already done with Debian kernels)
> >   and to include https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3aed3e2
> >   (applied upstream since v5.2)
> 
> Recent u-boot adds an "errata 80000688A fix" for Micrel KSZ8061 - 
> perhaps relevant for this issue...?
> 
> https://gitlab.denx.de/u-boot/u-boot/-/commit/baafd99

Seems this new errata fix is irrelevant, as the LIME2rev.H uses Micrel 
KSZ9031

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #111 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: Dominique Dumont <dod@debian.org>, debian-arm@lists.debian.org

Cc: 911560@bugs.debian.org, Vagrant Cascadian <vagrant@debian.org>

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Wed, 22 Jul 2020 09:53:42 +0200

[Message part 1 (text/plain, inline)]

Quoting Dominique Dumont (2020-02-16 19:15:21)
> On Tuesday, 11 February 2020 14:16:22 CET Dominique Dumont wrote:
> > This is promising:
> > 
> > https://lists.denx.de/pipermail/u-boot/2016-March/249735.html
> 
> Or not :-(
> 
> With this setup (and no GMAC_TX_DELAY), the network starts fine on rev Olimex 
> LIME2 rev G.2 but not on rev K.

Any news on proposing your bugfix to mainline u-boot, Dominique?

Wiki page https://linux-sunxi.org/Olimex_A20-OLinuXino-Lime2#GMAC_quirks 
now updated to cover the knowledge shared here and other places I could 
locate.

How did you reach the value "3" for the bugfix?  Did you test multiple 
values or followed advice from somewhere and could confirm that it was 
an improvement?  Reason I ask is that the only places I could find that 
specific value was at the wiki page and here in this bugreport, and (as 
I have now clarified) the information seems to not be well tested and is 
different from what Olimex use in their fork of u-boot.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #116 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dominique Dumont <dod@debian.org>

To: debian-arm@lists.debian.org, 911560@bugs.debian.org

Cc: 911560@bugs.debian.org, Vagrant Cascadian <vagrant@debian.org>, Jonas Smedegaard <jonas@jones.dk>

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Wed, 22 Jul 2020 13:38:21 +0200

On Wednesday, 22 July 2020 09:53:42 CEST Jonas Smedegaard wrote:
> Any news on proposing your bugfix to mainline u-boot, Dominique?

I've seen that this issue and the fix I've applied were discussed extensively 
back in 2016 in this thread:
- https://lists.denx.de/pipermail/u-boot/2016-March/248498.html

The conclusion of the thread above was to add 
CONFIG_RTL8211X_PHY_FORCE_MASTER config parameter. Which was done with :
https://lists.denx.de/pipermail/u-boot/2016-March/249734.html

So I did not create a pull request.

Note that setting CONFIG_RTL8211X_PHY_FORCE_MASTER parameter did 
not fix my Lime2 rev K board:
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911560#96

> How did you reach the value "3" for the bugfix?  Did you test multiple
> values or followed advice from somewhere and could confirm that it was
> an improvement?  

I used the value shown at the beginning of this bug report:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911560#15

I've not tested other values.

Since then, I've been side-tracked by some career changes [1] and did not 
investigate further this issue.

All the best

[1] https://www.linkedin.com/posts/dominique-dumont-grenoble_dominique-dumont-code-straight-activity-6689877896800079872-a1j3

Message #121 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dominique Dumont <dod@debian.org>, debian-arm@lists.debian.org

Cc: 911560@bugs.debian.org, Vagrant Cascadian <vagrant@debian.org>

Subject: Re: Bug#911560: [Kinda solved] Re: Network not working on Olimex LIME2 rev K ??

Date: Wed, 22 Jul 2020 15:17:03 +0200

[Message part 1 (text/plain, inline)]

Quoting Dominique Dumont (2020-07-22 13:38:21)
> On Wednesday, 22 July 2020 09:53:42 CEST Jonas Smedegaard wrote:
> > Any news on proposing your bugfix to mainline u-boot, Dominique?
> 
> I've seen that this issue and the fix I've applied were discussed 
> extensively back in 2016 in this thread:
> - https://lists.denx.de/pipermail/u-boot/2016-March/248498.html
> 
> The conclusion of the thread above was to add 
> CONFIG_RTL8211X_PHY_FORCE_MASTER config parameter. Which was done with :
> https://lists.denx.de/pipermail/u-boot/2016-March/249734.html
> 
> So I did not create a pull request.

Interesting thread but [seemingly about rev. C], not rev. G or rev. K.

[seemingly about rev. C]: Michael Haas mentions at 
https://lists.denx.de/pipermail/u-boot/2016-March/249087.html that his 
data makes sense when compared with RTL8211CL datasheept and not with 
RTL8211E datasheet, and LIME2 rev. C is the only officially released 
LIME2 revision with RTL8211CL PHY according to 
https://github.com/OLIMEX/OLINUXINO/blob/master/HARDWARE/A20-OLinuXino-LIME2/hardware_revision_changes_log.txt


> Note that setting CONFIG_RTL8211X_PHY_FORCE_MASTER parameter did 
> not fix my Lime2 rev K board:
> - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911560#96

CONFIG_RTL8211X_PHY_FORCE_MASTER only affects rev. C, not G2 or K since 
u-boot v2017.01: https://gitlab.denx.de/u-boot/u-boot/-/commit/cebf3f5

I have now added a list of known unexplored workarounds/fixes to 
https://linux-sunxi.org/Olimex_A20-OLinuXino-Lime2#GMAC_quirks


> > How did you reach the value "3" for the bugfix?  Did you test multiple
> > values or followed advice from somewhere and could confirm that it was
> > an improvement?  
> 
> I used the value shown at the beginning of this bug report:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=911560#15
> 
> I've not tested other values.

Thanks, that is helpful.


> Since then, I've been side-tracked by some career changes [1] and did 
> not investigate further this issue.
> 
> All the best
> 
> [1] https://www.linkedin.com/posts/dominique-dumont-grenoble_dominique-dumont-code-straight-activity-6689877896800079872-a1j3

Oh, exciting!  Enjoy your new life as independent.

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #126 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dieter <dieter@sarina-ist-doof.de>

To: 911560@bugs.debian.org

Subject: Tests for GMAC_TX_DELAY = ... with RevC A20 OLinuxino Lime 2

Date: Sun, 2 Aug 2020 12:29:45 +0200

[Message part 1 (text/plain, inline)]

Hello all!

I've made some tests with my Rev C A20 OLinuxino Lime 2 board.
So it uses the RTL8211C Phy.

source used: 2019.01+dfsg-7

The results are somewhat inconclusive i would say.
The board seems to be able to transfer data with TX_DELAY values up to 3.
Retry-counts went up when increasing the delay value.

Also attached are tests with a reversed data-flow, as well as test with
a longer cable.
Default cable: 1m
Long cable: 5m.


What i did not test: if DHCP would work with TX_DELAY=?, as i used
static ip addresses



I did some testing with the RevK board i have at my other place,
TX_DELAY = 3 was the needed value to get good ethernet speeds with that
board.

I can do some more testing if needed!

Best regards,
Dieter

[iperf3-RevC-TXDelay.7z (application/x-7z-compressed, attachment)]

Message #131 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dieter <dieter@sarina-ist-doof.de>

Subject: Re: Bug#911560: Tests for GMAC_TX_DELAY = ... with RevC A20 OLinuxino Lime 2

Date: Sun, 02 Aug 2020 12:54:36 +0200

[Message part 1 (text/plain, inline)]

Hi Dieter,

Quoting Dieter (2020-08-02 12:29:45)
> I've made some tests with my Rev C A20 OLinuxino Lime 2 board.
> So it uses the RTL8211C Phy.
> 
> source used: 2019.01+dfsg-7
> 
> The results are somewhat inconclusive i would say.
> The board seems to be able to transfer data with TX_DELAY values up to 3.
> Retry-counts went up when increasing the delay value.
> 
> Also attached are tests with a reversed data-flow, as well as test with
> a longer cable.
> Default cable: 1m
> Long cable: 5m.
> 
> 
> What i did not test: if DHCP would work with TX_DELAY=?, as i used
> static ip addresses
> 
> 
> 
> I did some testing with the RevK board i have at my other place,
> TX_DELAY = 3 was the needed value to get good ethernet speeds with that
> board.
> 
> I can do some more testing if needed!

Thanks for the above, and for offering to test more!

I have tried to summarize the kinds of test I imagine relevant, at 
https://linux-sunxi.org/Olimex_A20-OLinuXino-Lime2#GMAC_quirks

Length of ethernet cable should _not_ be relevant - as long as you stay 
within specs of 100 meter: 
https://en.wikipedia.org/wiki/Ethernet_over_twisted_pair#Variants

DHCP should be irrelevant to test - that's a higher-level protocol on 
top of tcp, and the problems are with packet loss at lower levels.

What _is_ relevant is which mode the PHY ends up being in - master or 
slave.  This can either be forced or auto-negotiated.  Beware that some 
releases of u-boot has it forced by default, and to fully test all 
combinations you will want to rebuild without that constraint.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #136 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dieter <dieter@sarina-ist-doof.de>

To: Jonas Smedegaard <jonas@jones.dk>, 911560@bugs.debian.org

Subject: Re: Bug#911560: Tests for GMAC_TX_DELAY = ... with RevC A20 OLinuxino Lime 2

Date: Sat, 8 Aug 2020 11:37:39 +0200

[Message part 1 (text/plain, inline)]

Hi Jonas,

as we found out here
(https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=845128#63),
FORCE_MASTER was disabled in 2019.01+dfsg-7.

I therefore downloaded u-boot-2020.07+dfsg-1 and did some testing.
System: debian buster


Results are attached.

What i did:

u-boot without FORCE_MASTER and different TX-Delays:
No difference can be seen i guess. I would have expected this, as the
FORCE_MASTER switch should only work for realtek-PHYs, and the REV K
uses the Micrel PHY.

However: I was not able to force the other partner (my old lenovo T500)
of the connection to a certain mode.
I thought ethtool would allow me to force the T500s, but i could not
find the option.

u-boot with FORCE_MASTER and TX-Delay 4:
Same as above.


u-boot with CONFIG_PHY_MICREL instead of CONFIG_PHY_REALTEK
Good performance with TX-Delay = [3,4].
The results with TX-Delay = 2 could not be reproduced.


Summed up: TX-delay = 3/4 seems useful for the Micrel phy.
With TX-delay = 0, no connection was possible at all.


Do you know what the switch regarding PHY_MICREL or PHY_REALTEK does? Is
this possibly only used in u-boot, and therefore irrelevant as soon as
linux boots?


As you mentioned this commit
(https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3aed3e2)
earlier in this thread, i would guess we should re-tests this with a
linux-image > 5.2?

Best regards,
Dieter

[u-boot-2020.07+dfsg-1.7z (application/x-7z-compressed, attachment)]

Message #141 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dieter <dieter@sarina-ist-doof.de>

Subject: Re: Bug#911560: Tests for GMAC_TX_DELAY = ... with RevC A20 OLinuxino Lime 2

Date: Sat, 08 Aug 2020 14:48:45 +0200

[Message part 1 (text/plain, inline)]

Quoting Dieter (2020-08-08 11:37:39)
> as we found out here
> (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=845128#63),
> FORCE_MASTER was disabled in 2019.01+dfsg-7.
> 
> I therefore downloaded u-boot-2020.07+dfsg-1 and did some testing.
> System: debian buster
> 
> 
> Results are attached.

Thanks!  Valuable!


> u-boot without FORCE_MASTER and different TX-Delays:
> No difference can be seen i guess. I would have expected this, as the
> FORCE_MASTER switch should only work for realtek-PHYs, and the REV K
> uses the Micrel PHY.

Right. That's why I omitted above test for rev. H-L boards in my list of 
TODOs at https://linux-sunxi.org/Olimex_A20-OLinuXino-Lime2#GMAC_quirks


> However: I was not able to force the other partner (my old lenovo T500)
> of the connection to a certain mode.
> I thought ethtool would allow me to force the T500s, but i could not
> find the option.

Whoa, setting master/slave mode was implemented only since Linux v5.7: 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bdbdac7649

Corresponding userspace support was introduced in ethtool v5.8: 
https://git.kernel.org/pub/scm/network/ethtool/ethtool.git/commit/?id=558f7cc33d

Ethtool v5.8 is not yet in Debian - was released upstream only 4 days 
ago.

Sorry, I thought master/slave was same as MDI/MDI-X, but those are 
independent: The former is which end provides clock source, then latter 
is which wires are used: 
https://en.wikipedia.org/wiki/Gigabit_Ethernet#1000BASE-T

According to 
https://www.speedguide.net/articles/network-adapter-optimization-3449 
when auto-negotiated "multi-port devices such as switches become the 
master when connected to a single port device. If both ends are 
multi-port devices, the one with higher seed bits becomes the master."

That seems to indicate that you should reliably put the device in slave 
mode by simply plugging it into a gigabit switch.  Still not sure how to 
force master mode (other than at the other end run Linux 5.7 and compile 
and use ethtool 5.8), as the above does not tell which end "wins" when 
both are single-port devices.


> u-boot with FORCE_MASTER and TX-Delay 4:
> Same as above.
> 
> 
> u-boot with CONFIG_PHY_MICREL instead of CONFIG_PHY_REALTEK

Ohhh, good point - I totally missed that detail!

Seems more optimal to enable CONFIG_PHY_MICREL_KSZ9031.

Would be helpful if you could test...

  * CONFIG_PHY_MICREL_KSZ9031 instead of CONFIG_PHY_MICREL
  * enabling both CONFIG_PHY_REALTEK and CONFIG_PHY_MICREL_KSZ9031
  * above, with various values for TX-Delay


> Good performance with TX-Delay = [3,4].
> The results with TX-Delay = 2 could not be reproduced.
> 
> 
> Summed up: TX-delay = 3/4 seems useful for the Micrel phy.
> With TX-delay = 0, no connection was possible at all.

I would expect TX-delay = 0 to behave same as TX-delay not set at all - 
is that your experience as well?


> Do you know what the switch regarding PHY_MICREL or PHY_REALTEK does? 
> Is this possibly only used in u-boot, and therefore irrelevant as soon 
> as linux boots?

those switches enable code chunks in u-boot.  My (vague!) understanding 
is that some but not all such code chunks does some initialization of 
hardware chips, and that Linux may or may not do its own 
(re)initialization of same bits.

In other words: Possibly - it depends... :-)


> As you mentioned this commit
> (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3aed3e2)
> earlier in this thread, i would guess we should re-tests this with a
> linux-image > 5.2?

Ideally we should test network quality from within u-boot, and if we 
identify some u-boot setup that fails from within u-boot but works from 
some Linux, then try identify what initialization the Linux code does 
and try figure out how that could be done in u-boot as well.

...because ideally we want u-boot to work reliably not only for 
initializing what Linux misses, but also for netbooting Linux.


Another test that would be helpful is if you test your board with u-boot 
built with A20-OLinuXino-Lime2-eMMC_defconfig (even if you do not have 
and/or use eMMC): My suspicion is that the added options enabled for 
that defconfig is harmless for non-eMMC boards, and knowing if in fact 
they are harmless is helpful to figure out how many binaries we really 
need to build in Debian, and if e.g. possible to say "use the -eMMC one 
for Micrel-based boards regardless of eMMC".


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #146 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dieter <dieter@sarina-ist-doof.de>

Subject: Re: Bug#911560: Tests for GMAC_TX_DELAY = ... with RevC A20 OLinuxino Lime 2

Date: Sat, 08 Aug 2020 15:46:15 +0200

[Message part 1 (text/plain, inline)]

Quoting Jonas Smedegaard (2020-08-08 14:48:45)
> Quoting Dieter (2020-08-08 11:37:39)
> > However: I was not able to force the other partner (my old lenovo 
> > T500) of the connection to a certain mode.
> > I thought ethtool would allow me to force the T500s, but i could not 
> > find the option.
> 
> Whoa, setting master/slave mode was implemented only since Linux v5.7: 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bdbdac7649
> 
> Corresponding userspace support was introduced in ethtool v5.8: 
> https://git.kernel.org/pub/scm/network/ethtool/ethtool.git/commit/?id=558f7cc33d
> 
> Ethtool v5.8 is not yet in Debian - was released upstream only 4 days 
> ago.

Request to upgrade of ethtool in debian: https://bugs.debian.org/968099


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #151 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Dieter <dieter@sarina-ist-doof.de>

To: Jonas Smedegaard <jonas@jones.dk>, 911560@bugs.debian.org

Subject: Re: Bug#911560: Tests for GMAC_TX_DELAY = ... with RevC A20 OLinuxino Lime 2

Date: Fri, 14 Aug 2020 09:03:22 +0200

Hi Jonas,


On 08/08/2020 14:48, Jonas Smedegaard wrote:

>> However: I was not able to force the other partner (my old lenovo T500)
>> of the connection to a certain mode.
>> I thought ethtool would allow me to force the T500s, but i could not
>> find the option.
> 
> Whoa, setting master/slave mode was implemented only since Linux v5.7: 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bdbdac7649
> 
> Corresponding userspace support was introduced in ethtool v5.8: 
> https://git.kernel.org/pub/scm/network/ethtool/ethtool.git/commit/?id=558f7cc33d
> 
> Ethtool v5.8 is not yet in Debian - was released upstream only 4 days 
> ago.
> Sorry, I thought master/slave was same as MDI/MDI-X, but those are 
> independent: The former is which end provides clock source, then latter 
> is which wires are used: 
> https://en.wikipedia.org/wiki/Gigabit_Ethernet#1000BASE-T
> 
> According to 
> https://www.speedguide.net/articles/network-adapter-optimization-3449 
> when auto-negotiated "multi-port devices such as switches become the 
> master when connected to a single port device. If both ends are 
> multi-port devices, the one with higher seed bits becomes the master."
> 
> That seems to indicate that you should reliably put the device in slave 
> mode by simply plugging it into a gigabit switch.  Still not sure how to 
> force master mode (other than at the other end run Linux 5.7 and compile 
> and use ethtool 5.8), as the above does not tell which end "wins" when 
> both are single-port devices.

Thanks for the information.
Since ethtool 5.8 is now available, i can install debian unstable on the
laptop and test different combinations.



>> u-boot with FORCE_MASTER and TX-Delay 4:
>> Same as above.
>>
>>
>> u-boot with CONFIG_PHY_MICREL instead of CONFIG_PHY_REALTEK
> 
> Ohhh, good point - I totally missed that detail!
> 
> Seems more optimal to enable CONFIG_PHY_MICREL_KSZ9031.
> 
> Would be helpful if you could test...
> 
>   * CONFIG_PHY_MICREL_KSZ9031 instead of CONFIG_PHY_MICREL
>   * enabling both CONFIG_PHY_REALTEK and CONFIG_PHY_MICREL_KSZ9031
>   * above, with various values for TX-Delay

Yes, i can test that!

>> Good performance with TX-Delay = [3,4].
>> The results with TX-Delay = 2 could not be reproduced.
>>
>>
>> Summed up: TX-delay = 3/4 seems useful for the Micrel phy.
>> With TX-delay = 0, no connection was possible at all.
> 
> I would expect TX-delay = 0 to behave same as TX-delay not set at all - 
> is that your experience as well?

To be honest, i did not disable the configuration, as i always started
from A20-OLinuXino..._defconfig, and there the Delay is set to 0 by default.

But i just checked, if the line with GMAC_TX_DELAY is commented out in
the config, make will ask for a value, 0 being the default.
-> Yes, TX-Delay 0 is equal to this config not being set at all.

>> Do you know what the switch regarding PHY_MICREL or PHY_REALTEK does? 
>> Is this possibly only used in u-boot, and therefore irrelevant as soon 
>> as linux boots?
> 
> those switches enable code chunks in u-boot.  My (vague!) understanding 
> is that some but not all such code chunks does some initialization of 
> hardware chips, and that Linux may or may not do its own 
> (re)initialization of same bits.
> 
> In other words: Possibly - it depends... :-)

I see, in that case i would suppose that network functionality in u-boot
might depend on the compiled in drivers.
Likely independent of network functionality when the OS is brought up.
I arrived at this conclusion, as u-boot without the MICREL Phy has
working ethernet in Linux with the TX-Delay being set.

>> As you mentioned this commit
>> (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3aed3e2)
>> earlier in this thread, i would guess we should re-tests this with a
>> linux-image > 5.2?
> 
> Ideally we should test network quality from within u-boot, and if we 
> identify some u-boot setup that fails from within u-boot but works from 
> some Linux, then try identify what initialization the Linux code does 
> and try figure out how that could be done in u-boot as well.
> 
> ...because ideally we want u-boot to work reliably not only for 
> initializing what Linux misses, but also for netbooting Linux.

I agree, i would propose to test the values for TX_DELAY from within
Linux, as there measuring upload / download performance is easier.
As soon as we find a good configuration there, we can test the
netbooting from u-boot.

-> Configuring a tftp-server on the laptop to serve the netboot images
to the OLinuXino should help here.

> Another test that would be helpful is if you test your board with u-boot 
> built with A20-OLinuXino-Lime2-eMMC_defconfig (even if you do not have 
> and/or use eMMC): My suspicion is that the added options enabled for 
> that defconfig is harmless for non-eMMC boards, and knowing if in fact 
> they are harmless is helpful to figure out how many binaries we really 
> need to build in Debian, and if e.g. possible to say "use the -eMMC one 
> for Micrel-based boards regardless of eMMC".
> 
> 
>  - Jonas
> 

You were right, the -eMMC boots find, one just needs to:
- either symlink the non-eMMC.dtb to the *-eMMC.dtb in /boot/dtbs...
- install the real -eMMC.dtb to /boot/dtbs...

Booting the device works fine in both cases, and i did not encounter any
error as of yet.
I wonder if flash-kernel will take care of updated the dtb in /boot upon
a kernel update.



Best regards,
Dieter

Message #156 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Dieter <dieter@sarina-ist-doof.de>

Subject: Re: Bug#911560: Tests for GMAC_TX_DELAY = ... with RevC A20 OLinuxino Lime 2

Date: Fri, 14 Aug 2020 12:58:31 +0200

[Message part 1 (text/plain, inline)]

Quoting Dieter (2020-08-14 09:03:22)
> On 08/08/2020 14:48, Jonas Smedegaard wrote:
> > Whoa, setting master/slave mode was implemented only since Linux 
> > v5.7: 
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bdbdac7649
> > 
> > Corresponding userspace support was introduced in ethtool v5.8: 
> > https://git.kernel.org/pub/scm/network/ethtool/ethtool.git/commit/?id=558f7cc33d
> > 
> > Ethtool v5.8 is not yet in Debian - was released upstream only 4 
> > days ago. Sorry, I thought master/slave was same as MDI/MDI-X, but 
> > those are independent: The former is which end provides clock 
> > source, then latter is which wires are used: 
> > https://en.wikipedia.org/wiki/Gigabit_Ethernet#1000BASE-T
> > 
> > According to 
> > https://www.speedguide.net/articles/network-adapter-optimization-3449 
> > when auto-negotiated "multi-port devices such as switches become the 
> > master when connected to a single port device. If both ends are 
> > multi-port devices, the one with higher seed bits becomes the 
> > master."
> > 
> > That seems to indicate that you should reliably put the device in 
> > slave mode by simply plugging it into a gigabit switch.  Still not 
> > sure how to force master mode (other than at the other end run Linux 
> > 5.7 and compile and use ethtool 5.8), as the above does not tell 
> > which end "wins" when both are single-port devices.
> 
> Thanks for the information.
> Since ethtool 5.8 is now available, i can install debian unstable on the
> laptop and test different combinations.

Yes - I was excited too seeing that package update appear today

I filed a bugreport requesting an update which possibly helped speed 
this up, but maybe it would have been done by now regardless.


> >> u-boot with FORCE_MASTER and TX-Delay 4:
> >> Same as above.
> >>
> >>
> >> u-boot with CONFIG_PHY_MICREL instead of CONFIG_PHY_REALTEK
> > 
> > Ohhh, good point - I totally missed that detail!
> > 
> > Seems more optimal to enable CONFIG_PHY_MICREL_KSZ9031.
> > 
> > Would be helpful if you could test...
> > 
> >   * CONFIG_PHY_MICREL_KSZ9031 instead of CONFIG_PHY_MICREL
> >   * enabling both CONFIG_PHY_REALTEK and CONFIG_PHY_MICREL_KSZ9031
> >   * above, with various values for TX-Delay
> 
> Yes, i can test that!

Thanks!

To be honest, I could test this myself as well - I do have those boards 
lying around, but some of them are in active use, and I keep finding 
other things more important/exciting, so really appreciate your help 
here!

Another benefit of this being done between the two of us is that we get 
the details more carefully laid out and documented.  At least for me it 
has been an eye-opener in just how complex a seemingly simple "LIME2 is 
flaky at high speeds" turns out to be, if we wanna do better than "then 
just avoid those flaky modes".


> >> Good performance with TX-Delay = [3,4].
> >> The results with TX-Delay = 2 could not be reproduced.
> >>
> >>
> >> Summed up: TX-delay = 3/4 seems useful for the Micrel phy.
> >> With TX-delay = 0, no connection was possible at all.
> > 
> > I would expect TX-delay = 0 to behave same as TX-delay not set at 
> > all - is that your experience as well?
> 
> To be honest, i did not disable the configuration, as i always started 
> from A20-OLinuXino..._defconfig, and there the Delay is set to 0 by 
> default.
> 
> But i just checked, if the line with GMAC_TX_DELAY is commented out in
> the config, make will ask for a value, 0 being the default.
> -> Yes, TX-Delay 0 is equal to this config not being set at all.

Thanks for confirming!


> >> Do you know what the switch regarding PHY_MICREL or PHY_REALTEK 
> >> does?
> >> Is this possibly only used in u-boot, and therefore irrelevant as 
> >> soon as linux boots?
> > 
> > those switches enable code chunks in u-boot.  My (vague!) 
> > understanding is that some but not all such code chunks does some 
> > initialization of hardware chips, and that Linux may or may not do 
> > its own (re)initialization of same bits.
> > 
> > In other words: Possibly - it depends... :-)
> 
> I see, in that case i would suppose that network functionality in 
> u-boot might depend on the compiled in drivers.

Yeah - except it is not really "drivers" in u-boot.


> Likely independent of network functionality when the OS is brought up.

Cold-booting u-boot is certainly independent of Linux.

But Warm-booting u-boot after rebooting from a loaded Linux is *not* 
independent of Linux.

Neither is Linux loaded from u-boot independent of u-boot.


> I arrived at this conclusion, as u-boot without the MICREL Phy has 
> working ethernet in Linux with the TX-Delay being set.

Imagine Micrel PHY needs pins X and Y enabled to work well, and u-boot 
sets X and flips Y, and Linux flips X and sets Y.

Cold-booting u-boot would fail, but then loading Linux would work.

Rebooting into u-boot from working linux would fail.

Rebooting into u-boot from failing u-boot would work.

Cold-booting u-boot, rebooting u-boot, then loading Linux would fail.

If however neither u-boot nor linux flips anything, only sets flags 
without probing first, then... (but they _need_ to probe _something_ 
e.g. to decide that the board has a Micrel chip at all, and we are in 
this mess because some of the probing and/or settings are undocumented).

In short: Possibly - It depends... :-)


> >> As you mentioned this commit 
> >> (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3aed3e2) 
> >> earlier in this thread, i would guess we should re-tests this with 
> >> a linux-image > 5.2?
> > 
> > Ideally we should test network quality from within u-boot, and if we 
> > identify some u-boot setup that fails from within u-boot but works 
> > from some Linux, then try identify what initialization the Linux 
> > code does and try figure out how that could be done in u-boot as 
> > well.
> > 
> > ...because ideally we want u-boot to work reliably not only for 
> > initializing what Linux misses, but also for netbooting Linux.
> 
> I agree, i would propose to test the values for TX_DELAY from within 
> Linux, as there measuring upload / download performance is easier.
> As soon as we find a good configuration there, we can test the 
> netbooting from u-boot.

Makes some sense, especially if you hope to "cut through" to a working 
setting without sifting through all possible combinations of options.

On the other hand, if the combinations to test is large it is more 
tedious to load both u-boot and linux.


> -> Configuring a tftp-server on the laptop to serve the netboot images 
> to the OLinuXino should help here.

Yes.

Simplest least reliable u-boot test seems to be to ping something.

Possibly slightly better might be to issue dhcp requests repeatedly, as 
I imagine that would send larger packets.
 
Most reliable (using only generally available u-boot commands) seems to 
be fetching something over tftp and then sending it back over tftp 
again, as that should measure actual transfer speed in either direction.


> > Another test that would be helpful is if you test your board with 
> > u-boot built with A20-OLinuXino-Lime2-eMMC_defconfig (even if you do 
> > not have and/or use eMMC): My suspicion is that the added options 
> > enabled for that defconfig is harmless for non-eMMC boards, and 
> > knowing if in fact they are harmless is helpful to figure out how 
> > many binaries we really need to build in Debian, and if e.g. 
> > possible to say "use the -eMMC one for Micrel-based boards 
> > regardless of eMMC".
> > 
> > 
> >  - Jonas
> > 
> 
> You were right, the -eMMC boots find, one just needs to:
> - either symlink the non-eMMC.dtb to the *-eMMC.dtb in /boot/dtbs...
> - install the real -eMMC.dtb to /boot/dtbs...

This trick should work:

  sudo FK_MACHINE='Olimex A20-OLinuXino-LIME2-eMMC' flash-kernel

...and then reboot (and cross your fingers and knock on wood)


> Booting the device works fine in both cases, and i did not encounter 
> any error as of yet.

Thanks for confirming.

I have thought some more on this, and adjusted my logic: In the past I 
thought "use the -eMMC if your board has eMMC soldered on" but now 
realize that the better way to look at it is this:

Use the -eMMC dtb for modern boards that *optionally* can hold an eMMC - 
i.e. the optimal dtb to use is the -eMMC one for rev. F and newer - 
regardless if your specific board has eMMC soldered on and regardless if 
you intend on using eMMC - the "eMMC" in the dtb name is simply a 
distinguishing trait of the board design.

By that new (to me) logic, it makes sense to include the force-master 
workaround only for the non-eMMC dtb: It is irrelevant for any of the 
two PHYs used in more modern boards.  Until now I was considering that 
to be an oversight.


> I wonder if flash-kernel will take care of updated the dtb in /boot 
> upon a kernel update.

flash-kernel will by default use the device-tree of the running kernel 
(but the "by default" might no longer apply, depending on how exactly 
you mess with symlinks or configfile contents).

So I recommend trying above trick.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #161 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Nicolas Dufresne <nicolas@ndufresne.ca>

To: 911560@bugs.debian.org

Subject: Testing Various delays

Date: Fri, 14 Aug 2020 14:26:30 -0400

As asked over IRC, I have tested various TX_DELAY on Lime2 rev K.

- 2, 3, and 4 was good enough to succeed a DHCP handshake.

- 0 and 1 made the request to never be seen over the network.

An obvious next step would be to test again with large file transfer,
perhaps a kernel image. I will report back.

regards,
Nicolas

Message #166 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Nicolas Dufresne <nicolas@ndufresne.ca>

To: 911560@bugs.debian.org

Subject: Re: Testing Various delays

Date: Fri, 14 Aug 2020 16:07:11 -0400

Follow up:

I've tested download rate with a 125MB file over tftp. Even though tftp
code (or tftp itself?) is inefficient I could repeat the following:

Delay   | 2   | 3   | 4   |  5  |
-------------------------------
Mb/s    | 2.0 | 2.1 | 2.7 | 2.1 |
Timeouts| 5   | 4   | 1   | 4   |

The timeouts are the number of time u-boot pinted a T during the
transfer. I've also attempted some test form Linux side, and notice
some asymmetry, 9.7 MB/s download, and 7.8 MB/s upload. That was with
the same file with scp.

I've then set the RX delay too, to 4, and could notice a very minimal
improvement, 8.5 MB/s upload. Now the issue is that these are all
terrible results for a gigabit ethernet. There must be other issues
around. But a delay of 4 at least makes it usable (at around 100Mb/s).

Le vendredi 14 août 2020 à 14:26 -0400, Nicolas Dufresne a écrit :
> As asked over IRC, I have tested various TX_DELAY on Lime2 rev K.
> 
> - 2, 3, and 4 was good enough to succeed a DHCP handshake.
> 
> - 0 and 1 made the request to never be seen over the network.
> 
> An obvious next step would be to test again with large file transfer,
> perhaps a kernel image. I will report back.
> 
> regards,
> Nicolas
> 

Message #171 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Nicolas Dufresne <nicolas@ndufresne.ca>

Subject: Re: Bug#911560: Testing Various delays

Date: Fri, 14 Aug 2020 23:37:53 +0200

[Message part 1 (text/plain, inline)]

Quoting Nicolas Dufresne (2020-08-14 22:07:11)
> Follow up:
> 
> I've tested download rate with a 125MB file over tftp. Even though 
> tftp code (or tftp itself?) is inefficient I could repeat the 
> following:
> 
> Delay   | 2   | 3   | 4   |  5  |
> -------------------------------
> Mb/s    | 2.0 | 2.1 | 2.7 | 2.1 |
> Timeouts| 5   | 4   | 1   | 4   |
> 
> The timeouts are the number of time u-boot pinted a T during the
> transfer. I've also attempted some test form Linux side, and notice
> some asymmetry, 9.7 MB/s download, and 7.8 MB/s upload. That was with
> the same file with scp.
> 
> I've then set the RX delay too, to 4, and could notice a very minimal
> improvement, 8.5 MB/s upload. Now the issue is that these are all
> terrible results for a gigabit ethernet. There must be other issues
> around. But a delay of 4 at least makes it usable (at around 100Mb/s).
> 
> Le vendredi 14 août 2020 à 14:26 -0400, Nicolas Dufresne a écrit :
> > As asked over IRC, I have tested various TX_DELAY on Lime2 rev K.
> > 
> > - 2, 3, and 4 was good enough to succeed a DHCP handshake.
> > 
> > - 0 and 1 made the request to never be seen over the network.
> > 
> > An obvious next step would be to test again with large file transfer,
> > perhaps a kernel image. I will report back.

Thanks a lot for these tests.  Really helpful!

There _are_ other limiting factors than ethernet speed involved - I do 
not know what maximum to expect on this kind of board, however.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #176 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Nicolas Dufresne <nicolas@ndufresne.ca>

Subject: Re: Bug#911560: Testing Various delays

Date: Fri, 14 Aug 2020 23:40:55 +0200

[Message part 1 (text/plain, inline)]

Quoting Nicolas Dufresne (2020-08-14 22:07:11)
> I've then set the RX delay too, to 4, and could notice a very minimal 
> improvement, 8.5 MB/s upload. Now the issue is that these are all 
> terrible results for a gigabit ethernet. There must be other issues 
> around. But a delay of 4 at least makes it usable (at around 100Mb/s).

Did you try setting RX delay to _other_ values?

Seems from adjustments of other hardware that don't need not be equal.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #181 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org

Subject: Re: Bug#911560: Tests for GMAC_TX_DELAY = ... with RevC A20 OLinuxino Lime 2

Date: Sat, 24 Oct 2020 05:03:38 +0200

[Message part 1 (text/plain, inline)]

Control: retitle -1 A20-OLinuXino-Lime2-eMMC Rev. K: very poor ethernet performance

Quoting Jonas Smedegaard (2020-08-08 15:46:15)
> Quoting Jonas Smedegaard (2020-08-08 14:48:45)
> > Quoting Dieter (2020-08-08 11:37:39)
> > > However: I was not able to force the other partner (my old lenovo 
> > > T500) of the connection to a certain mode. I thought ethtool would 
> > > allow me to force the T500s, but i could not find the option.
> > 
> > Whoa, setting master/slave mode was implemented only since Linux 
> > v5.7: 
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bdbdac7649
> > 
> > Corresponding userspace support was introduced in ethtool v5.8: 
> > https://git.kernel.org/pub/scm/network/ethtool/ethtool.git/commit/?id=558f7cc33d
> > 
> > Ethtool v5.8 is not yet in Debian - was released upstream only 4 
> > days ago.
> 
> Request to upgrade of ethtool in debian: 
> https://bugs.debian.org/968099

ethtool 5.8 is in Debian testing since late August.

With that version, simply running "ethtool eth0" shows if the card is 
currently running in master or slave mode - which is helpful to mention 
when doing calibration tests, as that possibly matters and it can change 
each time connection is negotiated.


Changing title of this bugreport to disambiguate from bug#916822.

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #188 received at 911560@bugs.debian.org (full text, mbox, reply):

From: "Andreas B. Mundt" <andi@debian.org>

To: 911560@bugs.debian.org

Subject: networking fails with A20-OLinuXino-Lime2-eMMC Rev. G2 and bullseye u-boot and -kernel

Date: Thu, 26 Aug 2021 20:01:16 +0200

Control: retitle -1 A20-OLinuXino-Lime2-(eMMC): Rev. K, G2: very poor/no ethernet performance


Hi all,

the last days, I tried upgrading two A20-OLinuXino-Lime2 boards to bullseye.
After booting the bullseye kernel, 5.10.46-4 (2021-08-03), the network did 
not work anymore: The interface shows up (IIRC, even a link is detected),
but any attempts to connect (DHCP) fail.  

After playing with different delays following [1], I succeeded with 
CONFIG_GMAC_TX_DELAY=4, where both boards seem to work fine. I also played a
bit with iperf3:  At higher bitrates (> approx. 200Mbits/s) more and more 
retries seem to be needed.  So the issue this bug is about seems to affect
also the G2 revision boards that worked fine so far.

Best regards,

  Andi


[1] https://wiki.debian.org/InstallingDebianOn/Allwinner#Olimex_A20-OLinuXino-LIME2_rev._K.3B_rev._G2_and_bullseye_kernel

Message #197 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org, Andreas B. Mundt <andi@debian.org>

Subject: Re: Bug#911560: networking fails with A20-OLinuXino-Lime2-eMMC Rev. G2 and bullseye u-boot and -kernel

Date: Thu, 26 Aug 2021 20:17:52 +0200

[Message part 1 (text/plain, inline)]

Control: retitle -1 A20-OLinuXino-Lime2-eMMC Rev. K: very poor ethernet performance

Hi Andreas,

Quoting Andreas B. Mundt (2021-08-26 20:01:16)
> Control: retitle -1 A20-OLinuXino-Lime2-(eMMC): Rev. K, G2: very poor/no ethernet performance

Please re-post your info about rev.G board to bug#927397 - thanks!


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #206 received at 911560-done@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560-done@bugs.debian.org

Subject: Re: Bug#911560: Testing Various delays

Date: Thu, 09 Jun 2022 12:23:53 +0200

[Message part 1 (text/plain, inline)]

Version: 2022.04+dfsg-1

This bug was fixed since u-boot 2022.04 - specifically in git commit
f11513d9: https://source.denx.de/u-boot/u-boot/-/commit/f11513d

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Message #211 received at 911560@bugs.debian.org (full text, mbox, reply):

From: Jonas Smedegaard <jonas@jones.dk>

To: 911560@bugs.debian.org

Subject: Re: Bug#911560: Testing Various delays

Date: Thu, 09 Jun 2022 12:33:17 +0200

[Message part 1 (text/plain, inline)]

Quoting Jonas Smedegaard (2022-06-09 12:23:53)
> Version: 2022.04+dfsg-1
> 
> This bug was fixed since u-boot 2022.04 - specifically in git commit
> f11513d9: https://source.denx.de/u-boot/u-boot/-/commit/f11513d

Correction - the actual fix was the later git commit 85da5587:
https://source.denx.de/u-boot/u-boot/-/commit/85da558

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.