XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    NVIDIA GPU passthrough on XCP-ng 8.3 fails after reboot — UUID/PCI ID changes

    Scheduled Pinned Locked Moved Hardware
    8 Posts 3 Posters 223 Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S Offline
      samuelolavo
      last edited by

      I’m trying to use passthrough for an NVIDIA GPU on XCP-ng 8.3.
      The host detects the GPU (lspci shows VGA + Audio), and IOMMU is enabled.

      However, whenever I apply xen-pciback.hide and reboot the host, XCP-ng generates a new internal UUID and new PCI ID for the GPU.
      As a result, I cannot even assign the GPU to a VM, because the device “changes” its reference after each reboot.

      Additional important context:

      I previously had a different GPU doing passthrough in a VM.

      That GPU was removed.

      The new GPU was installed in a different slot.

      I suspect the issues may be related to leftover metadata from the previous GPU, which prevents the new GPU from being recognized consistently for passthrough.

      Question:
      Has anyone encountered this problem? Is there a way to completely clear the old passthrough state on the host or ensure that the new GPU is recognized consistently for passthrough?

      1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Hi,

        Clear entirely the boot parameter with /opt/xensource/libexec/xen-cmdline --delete-dom0 xen-pciback.hide, reboot, and re-assign.

        S 1 Reply Last reply Reply Quote 0
        • S Offline
          samuelolavo @olivierlambert
          last edited by samuelolavo

          @olivierlambert

          Thank you for your reply.

          I followed your instructions; however, when I passthrough the GPU NVIDIA on the XenOrchestra and he same error occurs when I reboot.

          Change the PCI and UUID of the graphics card to the one before rebooting.

          S 1 Reply Last reply Reply Quote 0
          • S Offline
            samuelolavo @samuelolavo
            last edited by

            This post is deleted!
            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by

              Can you tell us on what platform you are doing this? It looks like a buggy PCI reset.

              S 1 Reply Last reply Reply Quote 0
              • S Offline
                samuelolavo @olivierlambert
                last edited by samuelolavo

                @olivierlambert

                I have a the XCP8.3 installed.
                Xen Orchestra, commit bcee5 .

                When I add it manually, the audio, for example, it asks for a reboot, but then, when I restart the server, it has another pci ID, and another ID.

                aba3b57a-fb08-4af6-9f05-3c07b4e494e8-image.png

                1 Reply Last reply Reply Quote 0
                • olivierlambertO Offline
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  Oo I'm not sure to follow. But let me add @Team-Hypervisor-Kernel in case that rings a bell.

                  1 Reply Last reply Reply Quote 0
                  • Y Offline
                    yannsionneau Vates 🪐 XCP-ng Team
                    last edited by

                    Hi,

                    @samuelolavo can you run the following command on the dom0 and paste its output please:

                    grep -A5 "'XCP-ng'" /etc/grub.cfg
                    

                    Then also run

                    lspci -vvv
                    xe pci-list
                    xl pci-assignable-list
                    
                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post