How to build new Qlogic Linux Drivers e.g. for former Broadcom 57800 NICs

schories

Dear XCP-ng team,

First of thank you for your hard work and patience. After upgrading from XenServer 7.2 to XCP-ng 7.4.1 everything works flawlessly, except the Quad (2x 1Gbit, 2x up to 10Gbit) NIC on a Dell R730XD:

exact model: QLogic 57800S Quad Port 2, 1GB x 2, 10Gb rNDC Base-T

issue:

eth0 and eth1 show no link
while at least eth2 and eth3 thankfully do

Firmware on NICs is the latest & greatest: 14.05.01

However, the driver in XenServer & XCP-ng is older: 7.14.6.0 (the version since XenServer 7.3.0)

Apparantly this issue is known to Citrix:

https://discussions.citrix.com/topic/394126-dell-r740-nic-bcm57800-110gb-ethernet-xen-74-link-problem/

Instead of blaming the hardware which came with the popular server - and which worked flawlessly until now, means with XenServer 7.2 that is - I'd like to solve the issue which I assume to be outdated drivers.

Dell just published the latest Qlogic network drivers for Linux - sources and binaries - in June 2018:

Sources:
https://downloads.dell.com/FOLDER04919194M/1/Source.tar

Binaries (RHEL6/RHEL7/SLES):
https://downloads.dell.com/FOLDER04919193M/1/Drivers.tar

As both XenServer and XCP-ng don't use the Kernel version from RHEL6 or RHEL7, but 4.4.0 in case of XCP-ng 4.7.1, those binaries won't help.

Afaik one has to use the XenServer DDK to build fitting binaries, correct?
Current DDK version on Citrix.com for XenServer 7.5.0 is still 7.3.0. No newer and no XCP-ng-specific DDK yet, correct?
What are the steps required to make sure not only I build the drivers for my personal use, but the updated driver sources are part of future XCP-ng builds? I'd like to help.

Thank you very much.

Kind regards,

Alexander

olivierlambert

I think @r1 is the man for asking this stuff

r1

Hehe I was going to reply this unread one but something came up and it got skipped later.

@schories What is the exact source for this/these driver(s)?

Usually you need a DDK and use its example driver to build for XenServer/XCP-NG.

I'll be happy to do it if pointed to right source package.

stormi

If it's just a matter of using a newer version, I'd try to identify which package in 7.5 provides the driver and try to update the RPM sources and spec file and rebuild them in https://github.com/xcp-ng/xenserver-build-env while making sure to use our builddeps yum repository (the xs repository from Citrix is from their development branch so there's no guarantee that it is suitable for building packages for 7.5): https://updates.xcp-ng.org/7/dev/builddeps/

Note that I never used the DDK so I may miss an important point about it being useful here.

schories

Thank you for all your kind and fast replies. I sincerely apologize for replying so late. Once I've built the drivers I'll make them available here.

olivierlambert

No worries! Eager to see the RPMs here

tim

Hey @schories, @stormi & @r1 - just wondering if anyone ever managed to build an RPM for these drivers?

Have a trio of Dell r630's sat here with the Broadcom/Qlogic 57800's that can't bring their 10GB-e links up on an XCP 7.5 install whereas they run perfectly under XS7.1

Thanks!
Tim

r1

@tim Let me try.

r1

@tim I'm trying to build Source/QC/Linux_Source/RHEL7/U4/src/qla2xxx-8.07.00.56.07.0-k. Is this the one you are looking for?

There is another directory in the Source which has following src rpms.

e4-open-iscsi-2.0-873.110.tar.gz  qlgc-fastlinq-8.24.2.0-1.rhel6u8.src.rpm  qlgc-fastlinq-8.24.2.0-1.rhel7u3.src.rpm  qlgc-fastlinq-8.24.2.0-1.sles12sp3.src.rpm
fastlinq-8.24.2.0.tar.bz2         qlgc-fastlinq-8.24.2.0-1.rhel6u9.src.rpm  qlgc-fastlinq-8.24.2.0-1.rhel7u4.src.rpm

If this worked on XS7.1, do you still have access to that host? and can you confirm the kernel module and its modinfo from there?

Generally, the process is straightforward as mentioned by @stormi but in this case the sources are too many to identify the right one.

tim

@r1 - So sorry had assumed I'd get an email notification of a reply!

Thanks for coming back; if that's a suitable driver for a Qlogic/Broadcom 57800 then it's perfect... shall do some more research.

I do still have access to XS7.1 hosts as they're in production currently. What details do you need?

Thanks!

r1

@tim I'd need output of lsmod and lspci -v from it.

tim

@r1 Thanks!

Here's lspci -v: https://www.dropbox.com/s/zwq72v3dk5gtpzc/lspci.txt?dl=0

and lsmod: https://www.dropbox.com/s/xkprap6b0vnn5sc/lsmod.txt?dl=0

tim

@r1 Quick note to say thanks for your earlier input but don't worry about this.

We've abandoned trying to get the Broadcom 57800 cards to run under XCP-NG or Xen 7.5; replaced them with Dell Intel i350&X540 Quad Port cards.

As a quick recap for anyone else with this issue, we had a set of 3 Dell R630 servers fitted with Dell's Qlogic/Broadcom 57800 based quad-port daughter network cards (2 x 10GBe RJ45's with 2 x 1GBe RJ45s). Hardware is rev 10 apparently and they identify within the OS as

Ethernet controller: Broadcom Limited NetXtreme II BCM57800 1/10 Gigabit Ethernet (rev 10)

filename: /lib/modules/4.4.0+2/updates/bnx2x.ko
version: 1.714.1

Under Xen 7.1 and earlier these ran perfectly, using the BNX2X driver v 1.714.1 along with the latest firmware from Dell. Oddly most of the reported issues seem to concern this version ?

The Broadcom/QLogic 57800 doesn't seem to be on Citrix's HCL any longer for newer releases so presumably someone's decided that the year old NICs are now obsolete?

Attempting to upgrade to Xen 7.2 or indeed XCP-NG 7.5 resulted in installation of a newer BNX2X driver (1.714.6 if memory serves), which for whatever reason was unable to bring the 10GBe links up or detect connected cables.

We attempted to update the driver further using the appropriate Citrix driver disks along with attempting firmware rollbacks to see if that was the problem.

I had been hoping to try a fresh build of the driver as above, but have neither the build environment nor knowledge to do that!

After burning through 3-4 days troubleshooting this, we've given up on the Broadcom cards and bought replacement Intel NICs - Dell i350&X540 quad port daughter cards, again offering 2 x 10GBe & 2 x 1GBe RJ45 ports. Swapped the cards over, booted the machines and upgraded successfully to XCP-NG on first attempt.

Zero hassle; zero issues.

I can only assume there's a fix to be had somewhere but we've not got the time to keep experimenting in the hope something eventually results in a previously working system resuming working!

If it helps anyone else, we bought the Intel cards from here: https://www.etb-tech.com/dell-intel-i350-x540-1-10gb-rj-45-sfp-quad-port-rndc-98493.html#.W5Dt5-hKi70