XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Multi gpu peer to peer not available in vm

    Scheduled Pinned Locked Moved XCP-ng
    4 Posts 2 Posters 21 Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • U Offline
      umberto
      last edited by

      Hi everyone,
      I am trying to get P2P communication working between two Nvidia L40s PCI GPU cards.
      I did this in qemu/kvm by adding the "x-nv-gpudirect-clique=1" flag to each -device entry in the VM spec. Performance wasn't great (~13GB/s in the cuda p2pBandwidthLatencyTest), but the p2p function seems to work.
      I can't achieve similar behavior in the latest xcp-ng 8.2 or 8.3 versions.

      Is it possible in xcp-ng? Has anyone struggled with such a problem?

      Thank you all in advance for your time and input!

      1 Reply Last reply Reply Quote 0
      • olivierlambertO Offline
        olivierlambert Vates 🪐 Co-Founder CEO
        last edited by

        Hi,

        I'm not entirely sure to get it. You want P2P working with 2x GPUs via PCI passthrough AND inside a VM?

        1 Reply Last reply Reply Quote 0
        • U Offline
          umberto
          last edited by

          exactly

          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            Hmm I'm not sure it's even possible due to the nature of isolation provided by Xen 🤔

            Let me ask @Team-Hypervisor-Kernel

            1 Reply Last reply Reply Quote 0
            • First post
              Last post