Add kernel boot params for dom0

Solemn

Hi,

To start with, I am using xen-server and xcp-ng in my home environment for a couple of years now, a big thanks to the devs for making this possible in a free version! Currently I am upgrading hardware as well as xen version and ran into an issue:

The new platform will be a Minisforum XTX780, Ryzen 7 7840HS. Installation worked out fine, but kern.log constantly logs errors:

Dec 18 20:14:11 mini kernel: [17703.715477] pcieport 0000:00:02.1: AER: Corrected error received: 0000:00:02.1
Dec 18 20:14:11 mini kernel: [17703.715489] pcieport 0000:00:02.1: PCIe Bus Error: severity=Corrected, type=Data Link Layer, (Receiver ID)
Dec 18 20:14:11 mini kernel: [17703.715506] pcieport 0000:00:02.1:   device [1022:14ee] error status/mask=00000040/00006000
Dec 18 20:14:11 mini kernel: [17703.715513] pcieport 0000:00:02.1:    [ 6] BadTLP

What I found so far is that it is somehow related to Active State POwer Management (ASPM) and can be avoided by adding boot kernel parameters "pcie_aspm=off"

Playing around a bit in the grub menu I actually managed to solve the error (at least no further errors have been logged) by adding the parameter to

menuentry 'XCP-ng' {
        search --label --set root root-heucgb
        multiboot2 /boot/xen.gz dom0_mem=2544M,max:2544M watchdog ucode=scan dom0_max_vcpus=1-8 crashkernel=256M,below=4G console=vga vga=mode-0x0311
        module2 /boot/vmlinuz-4.19-xen root=LABEL=root-heucgb ro nolvm hpet=disable console=hvc0 console=tty0 quiet vga=785 splash plymouth.ignore-serial-consoles **append pcie_aspm=of**
        module2 /boot/initrd-4.19-xen.img
}

When trying to make this a permanent boot parameter, I found that in EFI times grub doesn't seem to work like I am used to and since I don't want to break my boot routine, what is the way to permanently append this to the boot entry?

Cheers,

Solemn

Solemn

Allright,

didn't think it would be that easy...

Since I didn't find /etc/default/grub nor update-grub -like commands nor any hints on where to look I just gave altering /boot/grub/efi/EFI/xenserver/grub.cfg a try. Simply by adding an entry

menuentry 'XCP-ng no aspm' {
        search --label --set root root-heucgb
        multiboot2 /boot/xen.gz dom0_mem=2544M,max:2544M watchdog ucode=scan dom0_max_vcpus=1-8 crashkernel=256M,below=4G console=vga vga=mode-0x0311
        module2 /boot/vmlinuz-4.19-xen root=LABEL=root-heucgb ro nolvm hpet=disable console=hvc0 console=tty0 quiet vga=785 splash plymouth.ignore-serial-consoles append pcie_aspm=off
        module2 /boot/initrd-4.19-xen.img
}

and making that the default entry seems to do the magic. No initrd recompile necessary apparently.

Solemn

olivierlambert

Adding @stormi in the loop to check if it sounds okay

stormi

The grub setup is rather simple and not very flexible. There's just one file to modify, as you found out (/etc/grub.cfg in BIOS mode, /etc/grub-efi.cfg in EFI mode, both being symbolic links to the actual file location).

You can add an entry to it, but there's a small chance this doesn't play well with scripts from either XenServer or ourselves which may want to update the file and get confused. It's usually better to just modify the existing entries, ideally using /opt/xensource/libexec/xen-cmdline, so that the file structure remains unchanged.