XCP-ng 8.3 and Dell R660 - crash during boot, halts remainder of installer process (bnxt_en?)

umbradark

Hi everyone,

Apologies for the earlier confusion about the NIC model — the actual problematic hardware is the Broadcom BCM57504 quad-port NIC built into some Dell PowerEdge R660 systems.

When attempting a clean install of XCP-ng 8.3 (June 2025 ISO) with this card enabled, the installer crashes or fails to detect any network interfaces.
The issue appears to stem from the older bnxt_en driver included in the installer.
Once the host is updated with the newer Broadcom driver (broadcom-bnxt-en-1.10.3_232.0.155.5-1.xcpng8.3), the system becomes completely stable and all ports enumerate correctly.

For reference, blacklisting the bnxt_en module during installation did not allow the installer to proceed — the only working method is described below.

Workaround / Installation Procedure

Step 1 — BIOS Pre-Install

Reboot and press F2 to enter BIOS setup → System Setup → Integrated Devices.
Disable only the Integrated Network Card 1 (the BCM57504 quad-port NIC).
- Embedded NIC1/NIC2 can remain enabled.
Save and exit BIOS.

Step 2 — Install XCP-ng 8.3

Boot from the June 2025 XCP-ng 8.3 installer ISO.
Proceed with a normal installation.
- With the BCM57504 disabled, the installer will complete normally using any other available NIC.

Step 3 — Update the Host (Critical)

After the first boot, run the following commands:

yum clean all
yum update -y

Confirm that the Broadcom driver has been updated to:

broadcom-bnxt-en-1.10.3_232.0.155.5-1.xcpng8.3.x86_64

This updated driver resolves the crash and detection issues.

Note:
Testing confirmed that using the F9 driver disk method during install does not resolve the issue — the installer still fails to enumerate the BCM57504 properly, even when the updated driver ISO is loaded before installation.
The only reliable method is to complete installation with the NIC disabled, perform a full yum update, and then re-enable the NIC afterward.

Step 4 — Re-enable the Integrated NIC

Shut down the host cleanly:
```
shutdown -h now
```
Re-enter BIOS (F2 → Integrated Devices) and re-enable:
```
Integrated Network Card 1 → Enabled
```
Save and power on.

At this point, the BCM57504 will enumerate correctly and the host will boot without any kernel panics or NIC detection issues.

Notes

This behavior is isolated to the BCM57504 and the older bnxt_en driver shipped with the June 2025 installer.
The issue does not occur once the updated driver is installed via yum update.
Other Broadcom adapters, such as the BCM57414, install normally.
Blacklisting the driver or loading a driver disk with F9 during install does not bypass the issue — the NIC must be disabled in BIOS until after updating.

Hopefully this helps anyone else running into the same issue with R660 systems and the Broadcom 57504 NIC.
Once the host is updated and the NIC is re-enabled, everything works as expected.

Pilow

@umbradark said in XCP-ng 8.3 and Dell R660 - crash during boot, halts remainder of installer process (bnxt_en?):

Blacklisting the driver during install does not bypass the issue — the installer still fails to proceed without the NIC disabled in BIOS.

any idea if there is a problem with BCM57416 NetXtreme-E Dual-Media 10G RDMA Ethernet Controller ?

I have two clusters of 3 hosts to upgrade to 8.3 soon

olivierlambert

Thanks for your feedback @umbradark

Let me reping the @Team-OS-Platform-Release so we can think about a good strategy to reduce the needed steps

umbradark

@olivierlambert Thanks so much for the quick response and for looping in the Team-OS-Platform-Release folks — I really appreciate it.

I’d be more than happy to test an updated installer ISO if one is released. There’s a good chance I’ll be setting up at least one more R660 with the same hardware configuration in the near future, so this would be a perfect opportunity to validate the fix.

Thanks again for all your help and attention on this.

stormi

I'm going to build an updated installer with an updated bnxt_en driver, as more and more servers require it.

thomasp

We are in the process of commissioning a batch of R770 with this broadcom nic.

@stormi , has a new installer been made available?

stormi

@thomasp I'm struggling so much to find the time to do it, and my colleagues are equally busy, but I'll try to make it happen this week!

stormi

The installer is built. We are testing it, then we will be able to provide it to you so you can test it in turn.

stormi

Hi! We have an updated installer for you in https://xcp-ng.org/blog/2025/12/18/december-2025-security-and-maintenance-updates-for-xcp-ng-8-3-lts/ !

stormi

Let us know if you have any issue with it. It's OK on our side, but I'll wait for some time before making is the new default download.

dcskinner

@stormi said in XCP-ng 8.3 and Dell R660 - crash during boot, halts remainder of installer process (bnxt_en?):

Hi! We have an updated installer for you in https://xcp-ng.org/blog/2025/12/18/december-2025-security-and-maintenance-updates-for-xcp-ng-8-3-lts/ !

If I am reading it right, the link is to the 20250606 installer. Have you replaced the 20250606 iso with an updated one with the same June date in the name or did you intend to link a 20251218 iso?

stormi

@dcskinner That was a mistake. Thanks for your vigilance, it's fixed!

stormi

The name is xcp-ng-8.3.0-20250606.2.iso because it's the same update level as back then... Only with two updated drivers.

dcskinner

@stormi

Great! Thanks! I will test it out on my Dell R7525 hosts this afternoon.

Andrew

@dcskinner The correct link is: xcp-ng-8.3.0-20250606.2.iso

@olivierlambert The December 2025 blog post has the wrong link...

stormi

@Andrew had the wrong link. I fixed it as soon as it was mentioned.

dcskinner

@stormi The new installer boots fine on my systems. I was able to update my hosts from 8.2.1 to 8.3. Huzzah!

Thanks for all the hard work!

olivierlambert

Yes!! Congrats for everyone (including you for the feedback @dcskinner !)