ISO modification with additional RPM for NIC

LennertvdBerg

I'm fairly new to XCP-NG and would like to build a custom ISO for XCP-NG where I can add an additional RPM for a Mellanox ConnectX-6 Lx 10/25GbE SFP28. The problem is that I don't have other NICs installed and I can't install XCP-NG 8.2.1 because it detects during installation that there's no NIC in the system. I can install XCP-NG 8.3 beta 2 as the drivers are included there. So, I would like to include the drivers for the Mellanox in the ISO so that during installation the process will automatically detect it and I can run the installation.

In xcp-ng-8.3.0-beta2, there's an additional mellanox-mlnxen-5.4_1.0.3.0-4.xcpng8.3.x86_64.rpm in the Packages/ directory. In xcp-ng-8.2.1-20231130, there is no mellanox-mlnxen*.rpm at all. I found two Mellanox RPMs at Koji;

mellanox-mlnxen-alt-5.4_1.0.3.0-1.xcpng8.2.x86_64.rpm (https://koji.xcp-ng.org/buildinfo?buildID=2620)
mellanox-mlnxen-alt-5.9_0.5.5.0-1.1.xcpng8.2.x86_64.rpm (https://koji.xcp-ng.org/buildinfo?buildID=2868)

I tried following the instructions for ISO modification mentioned in the XCP-NG ISO modification documentation

First, I extracted the ISO using the following commands:

mkdir tmpmountdir/
mount -o loop filename.iso tmpmountdir/ # as root
cp -a tmpmountdir/. iso
umount tmpmountdir/ # as root
chmod a+w iso/ -R

Then, I used wget to download the RPMs into the Packages/ directory. After this, I updated the repodata/ using the following command (remember to install createrepo-c first):"

sudo apt install createrepo-c
rm repodata/ -rf
createrepo_c . -o .

Finally, I built the ISO using the instructions given in the XCP-NG documentation:

#OUTPUT=/path/to/destination/iso/file # change me
OUTPUT=/home/xcp-ng/new_iso/xcp-ng-8.2.1-20231130-mod.iso
VERSION=8.2 # change me
genisoimage -o $OUTPUT -v -r -J --joliet-long -V "XCP-ng $VERSION" -c boot/isolinux/boot.cat -b boot/isolinux/isolinux.bin \
            -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e boot/efiboot.img -no-emul-boot .
isohybrid --uefi $OUTPUT

However, when I use this ISO, the Mellanox ConnectX-6 Lx drivers do not load during installation.

Also, I have seen on the Nvidia website that new drivers for the ConnectX-6 Lx are available for Citrix XenServer Host 8.2 in version mlnx-en-23.10-2.1.3.1-xenserver8.2-x86_64.

So my questions are:

What am I doing wrong with building the ISO and including the RPMs?
Is it possible to include the mlnx-en-23.10-2.1.3.1-xenserver8.2-x86_64 for XCP-NG 8.2?
What steps do I need to take, and how?

Danp

Have you tried booting with the alternate kernel? https://docs.xcp-ng.org/installation/hardware/#during-system-installation

Did you read the docs about creating a custom ISO? https://docs.xcp-ng.org/project/development-process/ISO-modification/

Edit: You could also press F9 during the install process to install custom network drivers.

olivierlambert

Adding @stormi in the loop

LennertvdBerg

@Danp, UPDATED: I tried booting with an alternate kernel in XCP-NG 8.2.1 and XCP-NG 8.3 beta 2, but it didn't load the Mellanox ConnectX-6 Lx 10/25GbE drivers.

Yes, I've read the documentation about creating a custom ISO and have detailed my procedure above. The only part I'm unsure about is this:
"you need to add new RPMs not just replace existing ones, they need to be pulled by another existing RPM as dependencies. If there's none suitable, you can add the dependency to the xcp-ng-deps RPM."
I couldn’t realize or understand this step.

LennertvdBerg

@stormi could you maybe advise what I'm doing wrong?

olivierlambert

@stormi is just back today, I'll come back to you soon

LennertvdBerg

@olivierlambert Thanks.

stormi

I haven't forgotten you, but have been busy. Still need help?

LennertvdBerg

@stormi Hi, some help is welcome Still haven’t found a solutions.

stormi

Do you absolutely need all in one ISO? Otherwise there's a rather simple solution with an unmodified ISO + a driver ISO. We even already provide one: https://updates.xcp-ng.org/isos/drivers/8.x/

LennertvdBerg

@stormi I thought it’s convenient to have all in one as it’s easy for installation. But I can check this options as well. So you recommend to extract the iso to a separate USB drive and load drivers from there?

stormi

Yes, without any network connectivity available, that's the main option.

Regarding modifying the installation ISO, you'd need two things:

modify install.img to put the driver in it. That's the easiest part and you can use https://github.com/xcp-ng/xcp/tree/master/scripts/iso-remaster (see the example script in samples/). We still need to update the online docs to mention this option (ping @yann). This will give you network connectivity during the installation, but won't install the driver on the system.
install the driver on the target system. If I had done things right, you could just add the RPM in a directory and modify an XML file. However I have not done this right so it's not possible at the moment. There are several options, but I think the simplest is:
- add the RPM in Packages/ and regenerate the repodata + modify the installer code in install.img so that it installs it (look up 'xcp-ng-deps' and add the package name next to it)

LennertvdBerg

@stormi I’ll be back on Wednesday (just short holiday now), I’ll try your advice and see how it works