PCI Passthrough back to dom0

l1c

I've been trying to do pci passthrough of an AMD 5600xt . I got it to work on a Windows VM (though I did not test its performance), but now I want to assign the pci back to dom0, and I can't seem to find a straight answer on the web.

I tried

# xl pci-assignable-remove -r <uuid>

based on xen documentation but that did not work

Out of frustration I did:

/opt/xensource/libexec/xen-cmdline --set-dom0 "xen-pciback.hide=()"

and when I checked

pci-assignable-list

I got

libxl: error: libxl_pci.c:427:libxl_device_pci_assignable_list: Looks like pciback driver not loaded

Does any one see this being an issue/problem later on?

What would be the proper command to give the pci back to dom0?

onur

@l1c I am not 100% sure but can you check your grub.cfg and see if the pciback.hide is still there?

/boot/efi/EFI/xenserver/grub.cfg

If it is there remove the text xen-pciback.hide=(xx:xx.X) . (assuming your system uses EFI)

But use this for the last resort, I am sure someone here will have better solution.

dthenot

/opt/xensource/libexec/xen-cmdline --delete-dom0 xen-pciback.hide should do the trick to remove the device from the pci-assignable list of devices at the next reboot.
dom0 rebooting will take control of the device.

l1c

I see. I suspected that line in the docs would be it.
In that case, may I suggest modifying the wording in the docs? The current wording suggests the opposite. At least it does to me. The "from" in the docs makes it read as if you are hiding all devices from dom0.

l1c

@dthenot Before this thread dies, I just wanted to ask one more thing.
How would you go about only giving back one pci device back dom0?

tony

You can use the following command to display all the PCI devices being passthrough, and then just set them up again without the one you want to remove.

/opt/xensource/libexec/xen-cmdline --get-dom0 xen-pciback.hide

dthenot

Exactly as @tony said, the option --delete-dom0 just remove the option from the booting command line given to dom0, if you need to remove a device you just need to remove it from the command when calling --set-dom0. I'll take a look at the doc, you can modify it too, we are always open to contribution .

l1c

Thanks for all the help everyone. Everyone here is always very helpful
How does one go about modifying the docs?

olivierlambert

There's a link in the bottom of every documentation page

l1c

Gotcha
Will do
Thanks for the help!

lotusdew

@olivierlambert : I was able to do a passthrough and assign to VM. But then I needed to detach it. I did
/opt/xensource/libexec/xen-cmdline --delete-dom0 xen-pciback.hide

then followed by

/opt/xensource/libexec/xen-cmdline --set-dom0 "xen-pciback.hide=(0000:03:00.0)(0000:03.00.1)(0000:03:00.2)"

I had subsequently assigned it to a VM. Then I

Then reboot. The first time I did it it worked fine.

However, now its not working anymore.

My boot config looks like

menuentry 'XCP-ng' {
search --label --set root root-fwztre
multiboot2 /boot/xen.gz dom0_mem=7584M,max:7584M watchdog ucode=scan dom0_max_vcpus=1-16 crashkernel=256M,below=4G console=vga vga=mode-0x0311
module2 /boot/vmlinuz-4.19-xen root=LABEL=root-fwztre ro nolvm hpet=disable console=hvc0 console=tty0 quiet vga=785 splash plymouth.ignore-serial-consoles xen-pciback.hide=(0000:03:00.0)(0000:03.00.1)(0000:03:00.2)
module2 /boot/initrd-4.19-xen.img

[16:07 xilinx-xcp ~]# xl pci-assignable-list
libxl: error: libxl_pci.c:427:libxl_device_pci_assignable_list: Looks like pciback driver not loaded

Deleting and recreating the pciback.hide is not solving the problem.

dthenot

@lotusdew Your second PCI address is wrong. You have a dot instead of a colon: 0000:03.00.1 -> 0000:03:00.1.

lotusdew

@dthenot THANK YOU !!