Test results for Dell Poweredge R770 with NVMe drives
-
@olivierlambert eager to test this new ISO, we have two XCP clusters in 8.2 that need upgrading in 8.3 with these cards :
- BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller
Do you think we would be impacted ?
These same servers also have I350 Gigabit Network Connectioncard (quad port)
-
I think it's only the Broadcom NIC
-
FYI we have the Broadcom NetXtreme Gigabit Ethernet (BCM5720) in Dell R660 and Broadcom Adv. Dual 10GBASE-T Ethernet in Dell r640.
Currently have no issues with ether nic in XCP-NG 8.3
-
This post is deleted! -
sorry for unwanted bold characters in previous post
I do confirm, 8.3 LTS installation boot sequence hungs with these two adapters:
Broadcom 57504 25G SFP28 Quad Port Adapter OCP 3.0 NIC
firmware version: 23.31.18.10 (Dell p/n 540-BCRX)Broadcom 57504 Quad Port 10/25GbE SFP28 Adapter, PCIe Full Height
firmware version: 23.31.18.10 (Dell p/n 540-BDCZ)
Leaving enabled only these ones, we have NO HUNGS during 8.3 boot:
Broadcom 5720 Dual Port 1GbE LOM
firmware version: 23.31.1 (Dell p/n 540-BDKD)Broadcom 5719 Quad Port 1GbE BASE-T Adapter PCIe Low Profile V2
firmware version: 23.31.1 (Dell p/n 540-BDRK)---- -
Yes, that's why our new ISO coming soon will get a more recent version of the driver, fixing the problem
Hang on! -
many thanks, to start our evaluation XCP-ng + XOA we wait new ISO
XCP-ng maximum hardware nic quantity (eth ports) per host is 16 ...correct ?
-
@olivierlambert said in Test results for Dell Poweredge R770 with NVMe drives:
Hang on!
no pun intended ?
-
No pun, just wait for everyone to get back from our yearly Vates internal event
-
Hi,
I'm having the exact same issue.
Dell R650xs
BOSS with M2 in RAID1
Broadcom 25Gbps NIC'sGlad to find out it's not an isolated issue. All is working fine in 8.2.1 in production for almost 2 years now, but of course 8.2.1 is out of service so I'm hoping the new ISO will be out soon.
thanks,
Marc -
-
@Danp
OK thx, will test that right away! -
Tested the newest iso (xcp-ng-8.3.0-20250606.2.iso) with Dell R770 and Broadcom 57508 100GbE QSFP56 Dual Port Adapter, PCIe Full Height
It still fails to recognize the NIC, relevant log lines:
[ 57.592461] Broadcom NetXtreme-C/E/S driver bnxt_en v1.10.3-232.0.155.5+
[ 61.400879] bnxt_en 0001:5c:00.0 (unnamed net_device) (uninitialized): Firmware not responding, rc: -16 status: 0x8000
[ 61.421312] bnxt_en: probe of 0001:5c:00.0 failed with error -16By the way, it works ok with 8.2.1 iso
-
That's interesting
Is your firmware fully up to date on the NIC? -
@olivierlambert Yes, the NIC firmware is the latest.
Strange that it works with 8.2.1 -
Relevant lines in 8.2.1 where it succeeds:
[ 12.242658] Broadcom NetXtreme-C/E driver bnxt_en v1.10.0-216.0.119.1
[ 12.274753] bnxt_en 0001:5c:00.0 eth0: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff010000, node addr 40:5b:7f:6e:22:00
[ 12.274767] bnxt_en 0001:5c:00.0: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link)
[ 12.306588] bnxt_en 0001:5c:00.1 eth1: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff000000, node addr 40:5b:7f:6e:22:01
[ 12.306599] bnxt_en 0001:5c:00.1: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link)
[ 15.100023] bnxt_en 0001:5c:00.1 side-1659-eth1: renamed from eth1
[ 15.117296] bnxt_en 0001:5c:00.0 side-3513-eth0: renamed from eth0
[ 16.878049] bnxt_en 0001:5c:00.0 eth0: renamed from side-3513-eth0
[ 16.970941] bnxt_en 0001:5c:00.1 eth1: renamed from side-1659-eth1
[ 20.216480] bnxt_en 0001:5c:00.1 eth1: NIC Link is Up, 100000 Mbps full duplex, Flow control: ON - receive & transmit
[ 20.216487] bnxt_en 0001:5c:00.1 eth1: FEC autoneg off encodings: None
[ 20.734402] bnxt_en 0001:5c:00.0 eth0: NIC Link is Up, 100000 Mbps full duplex, Flow control: none
[ 20.734409] bnxt_en 0001:5c:00.0 eth0: FEC autoneg off encodings: None -
Not entirely strange, a driver version can make the difference.
-
Also works with alt driver (broadcom-bnxt-en-alt-1.10.3_231.0.162.0-1.xcpng8.2.x86_64) in 8.2.1:
# yum install broadcom-bnxt-en-alt # reboot # modinfo bnxt_en filename: /lib/modules/4.19.0+1/override/bnxt_en.ko version: 1.10.3-231.0.162.0 description: Broadcom NetXtreme-C/E/S network driver license: GPL srcversion: 533BB7E5866E52F63B9ACCB alias: pci:v000014E4d0000D800sv*sd*bc*sc*i* ... alias: pci:v000014E4d00001604sv*sd*bc*sc*i* depends: devlink,ipv6 retpoline: Y name: bnxt_en vermagic: 4.19.0+1 SMP mod_unload modversions# dmesg|grep bnxt_en [ 15.047312] Broadcom NetXtreme-C/E/S driver bnxt_en v1.10.3-231.0.162.0 [ 15.048046] bnxt_en 0001:5c:00.0 (unnamed net_device) (uninitialized): Device requests max timeout of 60 seconds, may trigger hung task watchdog [ 15.215453] bnxt_en 0001:5c:00.0 eth0: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff010000, node addr 40:5b:7f:6e:22:00 [ 15.215469] bnxt_en 0001:5c:00.0: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link) [ 15.216606] bnxt_en 0001:5c:00.1 (unnamed net_device) (uninitialized): Device requests max timeout of 60 seconds, may trigger hung task watchdog [ 15.265361] bnxt_en 0001:5c:00.1 eth1: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff000000, node addr 40:5b:7f:6e:22:01 [ 15.265372] bnxt_en 0001:5c:00.1: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link) [ 15.350702] bnxt_en 0001:5c:00.1 side-5183-eth1: renamed from eth1 [ 15.385121] bnxt_en 0001:5c:00.0 side-3345-eth0: renamed from eth0 [ 17.400142] bnxt_en 0001:5c:00.0 eth0: renamed from side-3345-eth0 [ 17.493640] bnxt_en 0001:5c:00.1 eth1: renamed from side-5183-eth1 [ 20.614447] bnxt_en 0001:5c:00.1 eth1: NIC Link is Up, 100000 Mbps (NRZ) full duplex, Flow control: none [ 20.614454] bnxt_en 0001:5c:00.1 eth1: FEC autoneg off encoding: None [ 21.113687] bnxt_en 0001:5c:00.0 eth0: NIC Link is Up, 100000 Mbps (NRZ) full duplex, Flow control: ON - receive & transmit [ 21.113694] bnxt_en 0001:5c:00.0 eth0: FEC autoneg off encoding: None
