XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    How to build new Qlogic Linux Drivers e.g. for former Broadcom 57800 NICs

    Scheduled Pinned Locked Moved Development
    13 Posts 5 Posters 4.9k Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • schoriesS Offline
      schories
      last edited by schories

      Dear XCP-ng team,

      First of thank you for your hard work and patience. After upgrading from XenServer 7.2 to XCP-ng 7.4.1 everything works flawlessly, except the Quad (2x 1Gbit, 2x up to 10Gbit) NIC on a Dell R730XD:

      exact model: QLogic 57800S Quad Port 2, 1GB x 2, 10Gb rNDC Base-T

      issue:

      • eth0 and eth1 show no link
      • while at least eth2 and eth3 thankfully do 🙂

      Firmware on NICs is the latest & greatest: 14.05.01

      However, the driver in XenServer & XCP-ng is older: 7.14.6.0 (the version since XenServer 7.3.0)

      Apparantly this issue is known to Citrix:

      https://discussions.citrix.com/topic/394126-dell-r740-nic-bcm57800-110gb-ethernet-xen-74-link-problem/

      Instead of blaming the hardware which came with the popular server - and which worked flawlessly until now, means with XenServer 7.2 that is - I'd like to solve the issue which I assume to be outdated drivers.

      Dell just published the latest Qlogic network drivers for Linux - sources and binaries - in June 2018:

      Sources:
      https://downloads.dell.com/FOLDER04919194M/1/Source.tar

      Binaries (RHEL6/RHEL7/SLES):
      https://downloads.dell.com/FOLDER04919193M/1/Drivers.tar

      As both XenServer and XCP-ng don't use the Kernel version from RHEL6 or RHEL7, but 4.4.0 in case of XCP-ng 4.7.1, those binaries won't help.

      1. Afaik one has to use the XenServer DDK to build fitting binaries, correct?

      2. Current DDK version on Citrix.com for XenServer 7.5.0 is still 7.3.0. No newer and no XCP-ng-specific DDK yet, correct?

      3. What are the steps required to make sure not only I build the drivers for my personal use, but the updated driver sources are part of future XCP-ng builds? I'd like to help. 🙂

      Thank you very much.

      Kind regards,

      Alexander

      1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        I think @r1 is the man for asking this stuff 🙂

        1 Reply Last reply Reply Quote 0
        • R Offline
          r1 XCP-ng Team
          last edited by

          Hehe 🙂 I was going to reply this unread one but something came up and it got skipped later.

          @schories What is the exact source for this/these driver(s)?

          Usually you need a DDK and use its example driver to build for XenServer/XCP-NG.

          I'll be happy to do it if pointed to right source package.

          1 Reply Last reply Reply Quote 0
          • stormiS Offline
            stormi Vates 🪐 XCP-ng Team
            last edited by stormi

            If it's just a matter of using a newer version, I'd try to identify which package in 7.5 provides the driver and try to update the RPM sources and spec file and rebuild them in https://github.com/xcp-ng/xenserver-build-env while making sure to use our builddeps yum repository (the xs repository from Citrix is from their development branch so there's no guarantee that it is suitable for building packages for 7.5): https://updates.xcp-ng.org/7/dev/builddeps/

            Note that I never used the DDK so I may miss an important point about it being useful here.

            1 Reply Last reply Reply Quote 0
            • schoriesS Offline
              schories
              last edited by

              Thank you for all your kind and fast replies. I sincerely apologize for replying so late. Once I've built the drivers I'll make them available here.

              1 Reply Last reply Reply Quote 1
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                No worries! Eager to see the RPMs here 🙂

                1 Reply Last reply Reply Quote 0
                • T Offline
                  tim
                  last edited by

                  Hey @schories, @stormi & @r1 - just wondering if anyone ever managed to build an RPM for these drivers?

                  Have a trio of Dell r630's sat here with the Broadcom/Qlogic 57800's that can't bring their 10GB-e links up on an XCP 7.5 install whereas they run perfectly under XS7.1 😕

                  Thanks!
                  Tim

                  1 Reply Last reply Reply Quote 0
                  • R Offline
                    r1 XCP-ng Team
                    last edited by

                    @tim Let me try.

                    1 Reply Last reply Reply Quote 0
                    • R Offline
                      r1 XCP-ng Team
                      last edited by

                      @tim I'm trying to build Source/QC/Linux_Source/RHEL7/U4/src/qla2xxx-8.07.00.56.07.0-k. Is this the one you are looking for?

                      There is another directory in the Source which has following src rpms.

                      e4-open-iscsi-2.0-873.110.tar.gz  qlgc-fastlinq-8.24.2.0-1.rhel6u8.src.rpm  qlgc-fastlinq-8.24.2.0-1.rhel7u3.src.rpm  qlgc-fastlinq-8.24.2.0-1.sles12sp3.src.rpm
                      fastlinq-8.24.2.0.tar.bz2         qlgc-fastlinq-8.24.2.0-1.rhel6u9.src.rpm  qlgc-fastlinq-8.24.2.0-1.rhel7u4.src.rpm
                      

                      If this worked on XS7.1, do you still have access to that host? and can you confirm the kernel module and its modinfo from there?

                      Generally, the process is straightforward as mentioned by @stormi but in this case the sources are too many to identify the right one.

                      1 Reply Last reply Reply Quote 0
                      • T Offline
                        tim
                        last edited by

                        @r1 - So sorry had assumed I'd get an email notification of a reply!

                        Thanks for coming back; if that's a suitable driver for a Qlogic/Broadcom 57800 then it's perfect... shall do some more research.

                        I do still have access to XS7.1 hosts as they're in production currently. What details do you need?

                        Thanks!

                        1 Reply Last reply Reply Quote 0
                        • R Offline
                          r1 XCP-ng Team
                          last edited by

                          @tim I'd need output of lsmod and lspci -v from it.

                          T 1 Reply Last reply Reply Quote 0
                          • T Offline
                            tim @r1
                            last edited by tim

                            @r1 Thanks!

                            Here's lspci -v: https://www.dropbox.com/s/zwq72v3dk5gtpzc/lspci.txt?dl=0

                            and lsmod: https://www.dropbox.com/s/xkprap6b0vnn5sc/lsmod.txt?dl=0

                            T 1 Reply Last reply Reply Quote 0
                            • T Offline
                              tim @tim
                              last edited by

                              @r1 Quick note to say thanks for your earlier input but don't worry about this.

                              We've abandoned trying to get the Broadcom 57800 cards to run under XCP-NG or Xen 7.5; replaced them with Dell Intel i350&X540 Quad Port cards.

                              As a quick recap for anyone else with this issue, we had a set of 3 Dell R630 servers fitted with Dell's Qlogic/Broadcom 57800 based quad-port daughter network cards (2 x 10GBe RJ45's with 2 x 1GBe RJ45s). Hardware is rev 10 apparently and they identify within the OS as

                              Ethernet controller: Broadcom Limited NetXtreme II BCM57800 1/10 Gigabit Ethernet (rev 10)

                              filename: /lib/modules/4.4.0+2/updates/bnx2x.ko
                              version: 1.714.1

                              Under Xen 7.1 and earlier these ran perfectly, using the BNX2X driver v 1.714.1 along with the latest firmware from Dell. Oddly most of the reported issues seem to concern this version ?

                              The Broadcom/QLogic 57800 doesn't seem to be on Citrix's HCL any longer for newer releases so presumably someone's decided that the year old NICs are now obsolete?

                              Attempting to upgrade to Xen 7.2 or indeed XCP-NG 7.5 resulted in installation of a newer BNX2X driver (1.714.6 if memory serves), which for whatever reason was unable to bring the 10GBe links up or detect connected cables.

                              We attempted to update the driver further using the appropriate Citrix driver disks along with attempting firmware rollbacks to see if that was the problem.

                              I had been hoping to try a fresh build of the driver as above, but have neither the build environment nor knowledge to do that!

                              After burning through 3-4 days troubleshooting this, we've given up on the Broadcom cards and bought replacement Intel NICs - Dell i350&X540 quad port daughter cards, again offering 2 x 10GBe & 2 x 1GBe RJ45 ports. Swapped the cards over, booted the machines and upgraded successfully to XCP-NG on first attempt.

                              Zero hassle; zero issues.

                              I can only assume there's a fix to be had somewhere but we've not got the time to keep experimenting in the hope something eventually results in a previously working system resuming working!

                              If it helps anyone else, we bought the Intel cards from here: https://www.etb-tech.com/dell-intel-i350-x540-1-10gb-rj-45-sfp-quad-port-rndc-98493.html#.W5Dt5-hKi70

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post