XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Test results for Dell Poweredge R770 with NVMe drives

    Scheduled Pinned Locked Moved Hardware
    26 Posts 7 Posters 1.9k Views 6 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M Offline
      mik
      last edited by

      in my first XCP-ng test server, I have this Broadcom network adapters (Dell R760):

      a) Broadcom NetXtreme Gigabit Ethernet (BCM5720)
      b) Broadcom Adv. Quad 25Gb Ethernet
      c) Broadcom Gigabit Ethernet BCM5719
      d) Broadcom BCM57504 4x25G SFP28 PCIE
      e) Broadcom Gigabit Ethernet BCM5719

      adapters at point a) ad b) are integrated (but I can try to disable them using BIOS),
      the other adapters can be disabled also extracting them (PCI).

      if useful I can do that and identify which one is causing 8.3 installation boot stuck

      1 Reply Last reply Reply Quote 0
      • P Online
        Pilow @olivierlambert
        last edited by

        @olivierlambert eager to test this new ISO, we have two XCP clusters in 8.2 that need upgrading in 8.3 with these cards :

        • BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller

        Do you think we would be impacted ?

        These same servers also have I350 Gigabit Network Connectioncard (quad port)

        1 Reply Last reply Reply Quote 0
        • olivierlambertO Online
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          I think it's only the Broadcom NIC

          1 Reply Last reply Reply Quote 0
          • A Online
            acebmxer
            last edited by

            FYI we have the Broadcom NetXtreme Gigabit Ethernet (BCM5720) in Dell R660 and Broadcom Adv. Dual 10GBASE-T Ethernet in Dell r640.

            Currently have no issues with ether nic in XCP-NG 8.3

            1 Reply Last reply Reply Quote 0
            • M Offline
              mik
              last edited by

              This post is deleted!
              1 Reply Last reply Reply Quote 0
              • M Offline
                mik
                last edited by

                sorry for unwanted bold characters in previous post

                I do confirm, 8.3 LTS installation boot sequence hungs with these two adapters:

                Broadcom 57504 25G SFP28 Quad Port Adapter OCP 3.0 NIC
                firmware version: 23.31.18.10 (Dell p/n 540-BCRX)

                Broadcom 57504 Quad Port 10/25GbE SFP28 Adapter, PCIe Full Height
                firmware version: 23.31.18.10 (Dell p/n 540-BDCZ)

                Leaving enabled only these ones, we have NO HUNGS during 8.3 boot:

                Broadcom 5720 Dual Port 1GbE LOM
                firmware version: 23.31.1 (Dell p/n 540-BDKD)

                Broadcom 5719 Quad Port 1GbE BASE-T Adapter PCIe Low Profile V2
                firmware version: 23.31.1 (Dell p/n 540-BDRK)----

                1 Reply Last reply Reply Quote 0
                • olivierlambertO Online
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  Yes, that's why our new ISO coming soon will get a more recent version of the driver, fixing the problem 🙂 Hang on!

                  P 1 Reply Last reply Reply Quote 0
                  • M Offline
                    mik
                    last edited by

                    many thanks, to start our evaluation XCP-ng + XOA we wait new ISO

                    XCP-ng maximum hardware nic quantity (eth ports) per host is 16 ...correct ?

                    1 Reply Last reply Reply Quote 0
                    • P Online
                      Pilow @olivierlambert
                      last edited by

                      @olivierlambert said in Test results for Dell Poweredge R770 with NVMe drives:

                      Hang on!

                      no pun intended ? 😃

                      1 Reply Last reply Reply Quote 0
                      • olivierlambertO Online
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        No pun, just wait for everyone to get back from our yearly Vates internal event 😉

                        1 Reply Last reply Reply Quote 0
                        • M Offline
                          MarcVdS
                          last edited by

                          Hi,
                          I'm having the exact same issue.
                          Dell R650xs
                          BOSS with M2 in RAID1
                          Broadcom 25Gbps NIC's

                          Glad to find out it's not an isolated issue. All is working fine in 8.2.1 in production for almost 2 years now, but of course 8.2.1 is out of service so I'm hoping the new ISO will be out soon.

                          thanks,
                          Marc

                          DanpD 1 Reply Last reply Reply Quote 0
                          • DanpD Offline
                            Danp Pro Support Team @MarcVdS
                            last edited by

                            @MarcVdS The new ISO is already available. See this post for more details.

                            M 1 Reply Last reply Reply Quote 0
                            • M Offline
                              MarcVdS @Danp
                              last edited by

                              @Danp
                              OK thx, will test that right away!

                              1 Reply Last reply Reply Quote 1
                              • Y Offline
                                yllar
                                last edited by yllar

                                Tested the newest iso (xcp-ng-8.3.0-20250606.2.iso) with Dell R770 and Broadcom 57508 100GbE QSFP56 Dual Port Adapter, PCIe Full Height

                                It still fails to recognize the NIC, relevant log lines:

                                [ 57.592461] Broadcom NetXtreme-C/E/S driver bnxt_en v1.10.3-232.0.155.5+
                                [ 61.400879] bnxt_en 0001:5c:00.0 (unnamed net_device) (uninitialized): Firmware not responding, rc: -16 status: 0x8000
                                [ 61.421312] bnxt_en: probe of 0001:5c:00.0 failed with error -16

                                By the way, it works ok with 8.2.1 iso

                                1 Reply Last reply Reply Quote 0
                                • olivierlambertO Online
                                  olivierlambert Vates 🪐 Co-Founder CEO
                                  last edited by

                                  That's interesting 🤔 Is your firmware fully up to date on the NIC?

                                  Y 1 Reply Last reply Reply Quote 0
                                  • Y Offline
                                    yllar @olivierlambert
                                    last edited by

                                    @olivierlambert Yes, the NIC firmware is the latest.
                                    Strange that it works with 8.2.1

                                    Y 1 Reply Last reply Reply Quote 0
                                    • Y Offline
                                      yllar @yllar
                                      last edited by

                                      Relevant lines in 8.2.1 where it succeeds:

                                      [ 12.242658] Broadcom NetXtreme-C/E driver bnxt_en v1.10.0-216.0.119.1
                                      [ 12.274753] bnxt_en 0001:5c:00.0 eth0: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff010000, node addr 40:5b:7f:6e:22:00
                                      [ 12.274767] bnxt_en 0001:5c:00.0: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link)
                                      [ 12.306588] bnxt_en 0001:5c:00.1 eth1: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff000000, node addr 40:5b:7f:6e:22:01
                                      [ 12.306599] bnxt_en 0001:5c:00.1: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link)
                                      [ 15.100023] bnxt_en 0001:5c:00.1 side-1659-eth1: renamed from eth1
                                      [ 15.117296] bnxt_en 0001:5c:00.0 side-3513-eth0: renamed from eth0
                                      [ 16.878049] bnxt_en 0001:5c:00.0 eth0: renamed from side-3513-eth0
                                      [ 16.970941] bnxt_en 0001:5c:00.1 eth1: renamed from side-1659-eth1
                                      [ 20.216480] bnxt_en 0001:5c:00.1 eth1: NIC Link is Up, 100000 Mbps full duplex, Flow control: ON - receive & transmit
                                      [ 20.216487] bnxt_en 0001:5c:00.1 eth1: FEC autoneg off encodings: None
                                      [ 20.734402] bnxt_en 0001:5c:00.0 eth0: NIC Link is Up, 100000 Mbps full duplex, Flow control: none
                                      [ 20.734409] bnxt_en 0001:5c:00.0 eth0: FEC autoneg off encodings: None

                                      1 Reply Last reply Reply Quote 0
                                      • olivierlambertO Online
                                        olivierlambert Vates 🪐 Co-Founder CEO
                                        last edited by

                                        Not entirely strange, a driver version can make the difference.

                                        1 Reply Last reply Reply Quote 0
                                        • Y Offline
                                          yllar
                                          last edited by yllar

                                          Also works with alt driver (broadcom-bnxt-en-alt-1.10.3_231.0.162.0-1.xcpng8.2.x86_64) in 8.2.1:

                                          # yum install broadcom-bnxt-en-alt
# reboot

# modinfo bnxt_en
filename:       /lib/modules/4.19.0+1/override/bnxt_en.ko
version:        1.10.3-231.0.162.0
description:    Broadcom NetXtreme-C/E/S network driver
license:        GPL
srcversion:     533BB7E5866E52F63B9ACCB
alias:          pci:v000014E4d0000D800sv*sd*bc*sc*i*
...
alias:          pci:v000014E4d00001604sv*sd*bc*sc*i*
depends:        devlink,ipv6
retpoline:      Y
name:           bnxt_en
vermagic:       4.19.0+1 SMP mod_unload modversions 

                                          # dmesg|grep bnxt_en
[   15.047312] Broadcom NetXtreme-C/E/S driver bnxt_en v1.10.3-231.0.162.0
[   15.048046] bnxt_en 0001:5c:00.0 (unnamed net_device) (uninitialized): Device requests max timeout of 60 seconds, may trigger hung task watchdog
[   15.215453] bnxt_en 0001:5c:00.0 eth0: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff010000, node addr 40:5b:7f:6e:22:00
[   15.215469] bnxt_en 0001:5c:00.0: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link)
[   15.216606] bnxt_en 0001:5c:00.1 (unnamed net_device) (uninitialized): Device requests max timeout of 60 seconds, may trigger hung task watchdog
[   15.265361] bnxt_en 0001:5c:00.1 eth1: Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/50Gb/100Gb/200Gb Ethernet found at mem 2affff000000, node addr 40:5b:7f:6e:22:01
[   15.265372] bnxt_en 0001:5c:00.1: 252.048 Gb/s available PCIe bandwidth (16 GT/s x16 link)
[   15.350702] bnxt_en 0001:5c:00.1 side-5183-eth1: renamed from eth1
[   15.385121] bnxt_en 0001:5c:00.0 side-3345-eth0: renamed from eth0
[   17.400142] bnxt_en 0001:5c:00.0 eth0: renamed from side-3345-eth0
[   17.493640] bnxt_en 0001:5c:00.1 eth1: renamed from side-5183-eth1
[   20.614447] bnxt_en 0001:5c:00.1 eth1: NIC Link is Up, 100000 Mbps (NRZ) full duplex, Flow control: none
[   20.614454] bnxt_en 0001:5c:00.1 eth1: FEC autoneg off encoding: None
[   21.113687] bnxt_en 0001:5c:00.0 eth0: NIC Link is Up, 100000 Mbps (NRZ) full duplex, Flow control: ON - receive & transmit
[   21.113694] bnxt_en 0001:5c:00.0 eth0: FEC autoneg off encoding: None
                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post