Intel X550T 2.5G Not Working

livegrenier

Hey,

I got an Intel Corporation Ethernet Controller 10G X550T, I'm trying to get 2.5Gb but XCP-NG keeps putting it to 1Gb

On the other port on that card, I have it connected to a 10G switch and that works fine.

I also use another port for the management that is on another card, that one is expected to be 1Gb

Any suggestions?

here's that nic's info out:

Settings for eth2:
	Supported ports: [ TP ]
	Supported link modes:   100baseT/Full 
	                        1000baseT/Full 
	                        10000baseT/Full 
	                        2500baseT/Full 
	                        5000baseT/Full 
	Supported pause frame use: Symmetric
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  100baseT/Full 
	                        1000baseT/Full 
	                        10000baseT/Full 
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Speed: 1000Mb/s
	Duplex: Full
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	Auto-negotiation: on
	MDI-X: Unknown
	Supports Wake-on: d
	Wake-on: d
	Current message level: 0x00000007 (7)
			       drv probe link
	Link detected: yes

Thanks.

alt text

livegrenier

Just wanted to confirm I got that TX401 NIC and it's been working well so far on 2.5G

Andrew

@livegrenier That is correct. Intel officially does not officially support 2.5/5G on the x550 cards.

You can manually add the 2.5/5G advertisement, but it may not work correctly as the card does not fully support the standard.

Intel Readme for the driver with ethtool example.

livegrenier

Hi, @Andrew,

Wow thanks for the info, I'll try to return my card then, any ideas what's the best card to do 2.5G on xcp-ng? All I'm seeing is people with compatibility problems when I try to research this topic

Thanks

tjkreidl

@livegrenier 2.5 GiB is unusual. Have used X5xx series NICss before, but only at 1 or 10 GiB.

Andrew

@livegrenier Good question... Why not use 10G on the Intel? That would be the best choice. The intel x540 (1/10G) and newer are good choices for 10G cards.

I think it's worth just trying to manually enable the 2.5G speed on the 550 card. Intel does not support some required features of the NBASE-T standard, but it should still work ok. The x550 might need a NVM firmware update (available from intel or OEM).

Other milti-gig PCIe cards:

The Intel i225/i226 seems to work well, but it's not a 'server' chipset. There are true intel PCIe card, and many cheaper clones using the i225 chip. This is becoming a more common on-board ethernet chip and the driver has good Linux community testing and support.

The r8125 will also be supported, it's not a great choice. Again, desktop chipset. These are cheap for a reason. This chip is supported by the (not so great) vendor driver and is kind of an odd duck that needs to supported to bring XCP to a larger user base on cheaper hardware.

The AQC chipsets are used in many cards. The AQC107/108 is on the HCL and supported with the current driver. This is the only officially supported 2.5 ethernet by XenServer (but not XCP!). The newer cards (AQC113/etc) should be supported in the new alt driver available for easy manual install (soon). The different chipset support some range of 10M/100M/1G/2.5G/5G/10G.

While you can make USB ethernet work, stay away from them. Just say NO to USB...

livegrenier

@Andrew I was hoping to do 2.5Gb because that's what my pfsense can do.

Do you have any suggestions on forcing the speed? I attempted several things and it didn't seem to work.

I did update my firmware since so maybe it will work now, but if you have a suggestion that will survive a reboot ( I believe ethtool is not persistent) let me know.

Thanks.

Andrew

@livegrenier ethtool is not persistent, you can add the command to run at startup.

Quoting the readme:

By default, devices based on the Intel(R) Ethernet Controller x550 do not
advertise 2.5 Gbps or 5 Gbps. To have your device advertise these speeds, use
the following:

# ethtool -s <ethX> advertise N

Where N is a combination of the following.
100baseTFull   0x008
1000baseTFull  0x020
2500baseTFull  0x800000000000
5000baseTFull  0x1000000000000
10000baseTFull 0x1000

For example, to turn on all modes:
# ethtool -s <ethX> advertise 0x1800000001028

So, use ethtool -s ethX advertise 0x1800000001028
You may have to restart negotiation after the change ethtool -r ethX

livegrenier

@Andrew unfortunately doesn't sound like a great option long term, you think this will work out of the box?

https://www.amazon.ca/-/en/Adaptateur-Ethernet-Gigabit-double-Windows/dp/B0C54C4SNX/ref=sr_1_5?__mk_fr_CA=ÅMÅŽÕÑ&crid=FQSUANBXGHRQ&keywords=i226&qid=1700940204&sprefix=i226+%2Caps%2C61&sr=8-5

I'm looking for something with great support that will work for a long time, my current card doesn't sound like this going to be that unfortunately

Andrew

@livegrenier Yes, the i226 should work, but that one is very expensive. Do you need two ports? The single port ones are cheap ($25us).

I have not tested one of the generic dual port i226 cards, I'll order one for testing (about $40us).

If you want an out-of-the-box known supported multi-gig card then you want an AQC107 ($64us, today).

05:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)

driver: atlantic
version: 2.0.3.0-kern
firmware-version: 3.1.86
expansion-rom-version:
bus-info: 0000:05:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

Settings for eth4:
        Supported ports: [ TP ]
        Supported link modes:   100baseT/Full
                                1000baseT/Full
                                10000baseT/Full
                                2500baseT/Full
                                5000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  100baseT/Full
                                1000baseT/Full
                                10000baseT/Full
                                2500baseT/Full
                                5000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        MDI-X: Unknown
        Link detected: yes

livegrenier

@Andrew I really appreciate your ideas, I like the idea of paying less for just 1 port, you are right I do not need 2 ports.

I'm in Montreal so I have to use amazon.ca, the 10G card you link is not available in Canada but for the same price as the 2.5 dual nic card I can get this one:

https://www.amazon.ca/Port-Gigabit-Ethernet-Network-Card/dp/B07G2DV9K2/ref=sr_1_2?crid=24N7U52FGEOL8&keywords=AQC107&qid=1700951323&sprefix=aqc107%2Caps%2C70&sr=8-2

That should work right? I just don't want to be returning a card every week

Andrew

@livegrenier Looks ok (with full height bracket). You might need to fix the heatsink on that card (check reviews).

The TPlink TX401 looks ok.

livegrenier

@Andrew The TX201 would not work right?

I think it needs special drivers and all of that? no out of the box support from what I can tell...

Andrew

@livegrenier The TP-Link TX201 uses the RTL8125 chipset.

livegrenier

@Andrew thanks, and I see the tx401 has a chipset AQC107, so I'll go with that if you think it's going to work out of the box, I'll probably put this project on hold for a moment, but as soon as I have a confirmation I'll update this post to let everyone know how it went, might take a few weeks but I will.

Thanks for the help

livegrenier

Just wanted to confirm I got that TX401 NIC and it's been working well so far on 2.5G

Nephtys

I have been able to run a script at startup that enables 2.5g on the Intel X550. Here's how :

Create the script that will run at startup

nano /usr/local/bin/x550.sh

Content of x550.sh :

#!/usr/bin/bash

sleep 1m
ethtool -s eth1 advertise 0x1800000001028
echo `date +%F" "%T` "Startup script worked" >> /root/x550.log

Make the script executable :

chmod +x /usr/local/bin/x550.sh

Create the service that runs the script at startup

nano /etc/systemd/system/x550.service

Content of x550.service :

[Unit]

Description=Runs /usr/local/bin/x550.sh

[Service]

ExecStart=/usr/local/bin/x550.sh

[Install]

WantedBy=multi-user.target

Enable the service

systemctl enable x550

Reboot and run this command to see if the link speed is 2.5g

ethtool ethX (replace X with yours)

It should show this result :

Supported ports: [ TP ]
       Supported link modes:   100baseT/Full
                               1000baseT/Full
                               10000baseT/Full
                               2500baseT/Full
                               5000baseT/Full
       Supported pause frame use: Symmetric
       Supports auto-negotiation: Yes
       Supported FEC modes: Not reported
       Advertised link modes:  100baseT/Full
                               1000baseT/Full
                               10000baseT/Full
                               2500baseT/Full
                               5000baseT/Full
       Advertised pause frame use: Symmetric
       Advertised auto-negotiation: Yes
       Advertised FEC modes: Not reported
       Speed: 2500Mb/s
       Duplex: Full
       Port: Twisted Pair
       PHYAD: 0
       Transceiver: internal
       Auto-negotiation: on
       MDI-X: Unknown
       Supports Wake-on: d
       Wake-on: d
       Current message level: 0x00000007 (7)
                              drv probe link
       Link detected: yes

You can check the log file of your script located at /root/x550.log

I used this article to create everything :
https://www.redhat.com/sysadmin/replacing-rclocal-systemd