XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Coral TPU PCI Passthrough

    Scheduled Pinned Locked Moved Compute
    38 Posts 7 Posters 7.8k Views 11 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R Offline
      redakula @andSmv
      last edited by

      @andSmv Thanks! 👍 😄

      I tried to be as uninvasive as possible and changed the symbolic link xen.gz to point to the xen.gz from the RPM you created.

      Unfortunately still the same error (It does seem to boot the xen from the RPM as this has version 4.17.3-3 vs. the one currently in the repos which has version 4.17.3-4).

      [2024-05-24 17:06:33] (XEN) [  674.051176] Domain 14 (vcpu#2) crashed on cpu#22:
[2024-05-24 17:06:33] (XEN) [  674.051178] ----[ Xen-4.17.3-3  x86_64  debug=n  Not tainted ]----
[2024-05-24 17:06:33] (XEN) [  674.051179] CPU:    22
[2024-05-24 17:06:33] (XEN) [  674.051180] RIP:    0010:[<ffffffffa8581584>]
[2024-05-24 17:06:33] (XEN) [  674.051180] RFLAGS: 0000000000000286   CONTEXT: hvm guest (d14v2)
[2024-05-24 17:06:33] (XEN) [  674.051182] rax: ffffbd9c00149800   rbx: ffff9e9247cc9000   rcx: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051182] rdx: 00000000fee77000   rsi: 0000000000000000   rdi: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051183] rbp: ffffbd9c00327690   rsp: ffffbd9c00327658   r8:  0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051183] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051184] r12: ffffbd9c003276ac   r13: 0000000000000011   r14: ffff9e92413390c0
[2024-05-24 17:06:33] (XEN) [  674.051185] r15: 0000000000000077   cr0: 0000000080050033   cr4: 0000000000750ef0
[2024-05-24 17:06:33] (XEN) [  674.051185] cr3: 0000000103806000   cr2: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051186] fsb: 00007b6e7a42a8c0   gsb: ffff9e925b500000   gss: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051186] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0018   cs: 0010

      It does appear that there is some movement upstream on this (if i interpret the xen mailing list correctly).
      This patch series references the same title as the patch in this thread from 2022 and a bunch of other related work:
      https://lore.kernel.org/xen-devel/cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com/

      andSmvA 1 Reply Last reply Reply Quote 0
      • andSmvA Offline
        andSmv Vates 🪐 XCP-ng Team Xen Guru @redakula
        last edited by

        @redakula
        Well, this was unfortunately one of the potential outcome. Unfortunately we don't have the hardware to make more "in deep" debug. I will talk to Marek next week (on Xen Summit) about this patch series and if we could expect it eventually fix the issue with Coral TPU.
        Will keep you posted.

        R 1 Reply Last reply Reply Quote 2
        • R Offline
          redakula @andSmv
          last edited by

          @andSmv

          Thanks 🙂
          Let me know and i will be happy to continue testing 👍

          1 Reply Last reply Reply Quote 0
          • R redakula referenced this topic on
          • N Offline
            Niall Con
            last edited by

            This post is deleted!
            1 Reply Last reply Reply Quote 0
            • N Offline
              Niall Con
              last edited by

              Hi.
              I'm just wondering if anyone ever got this to work? It's been a long time since I trialled xcp-ng. I really wanted to use it use the coral pci tpu for a Frigate NVR solution.
              The coral.ai website only provides driver installation instructions for debian based systems.

              I purchased a dual edge M.2 version and put it in the M.2 that was being used by the Wifi card (which I don't need.
              This went into a mini pc running a Ryzen 5800u.

              When I lspci -nn | grep 089a looking to see if the host machine recognises it. I get the following:

              03:00.0 Non-VGA unclassified device [0000]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

              So it does see it. But when I look in Xen Orchestra, there's no sign.

              Was anyone successful in getting this to work?

              Thank you.

              R 1 Reply Last reply Reply Quote 0
              • R Offline
                redakula @Niall Con
                last edited by

                @Niall-Con

                Nope - i tested recently with the latest updates and the vm still crashes...

                There were some patches for Xen and i tested the one @andSmv posted without any change.
                There looked to be quite a few patches around similar issues on upstream Xen but i have no idea if any of them were relevant.

                Maybe we will see improvement once Xcp-ng 9 comes about with a more recent version?

                N 1 Reply Last reply Reply Quote 0
                • olivierlambertO Offline
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  Ping @andSmv I don't remember the status of this

                  1 Reply Last reply Reply Quote 0
                  • N Offline
                    Niall Con @redakula
                    last edited by

                    @redakula Thanks for letting me know.
                    I guess I'll just have to go with a dedicated low power stand alone device for Frigate and keep an eye out in the future to see if it makes it's way into Xcp-ng 9.

                    1 Reply Last reply Reply Quote 0
                    • andSmvA Offline
                      andSmv Vates 🪐 XCP-ng Team Xen Guru
                      last edited by

                      I don't aware if there's something new from Marek who initially worked on these patches. I think at the time he addressed not this particular hardware but rather the global issue, and this patch wasn't tested with Coral HW, so most probably that's why it doesn't work (may be more issues...)

                      I will ping Marek on XEN Community Matrix channel to know if there's something new at that level and will keep you posted.

                      M 1 Reply Last reply Reply Quote 1
                      • M Offline
                        milch @andSmv
                        last edited by

                        @andSmv Did you ever hear back by any chance?

                        andSmvA 2 Replies Last reply Reply Quote 0
                        • andSmvA Offline
                          andSmv Vates 🪐 XCP-ng Team Xen Guru @milch
                          last edited by

                          Just got the answer from Marek on that. The patches he made was tested with Intel Wifi cards and was targeting the similar issue (MSI-x table) but not the same as Coral TPU (PBA). It should be not very very difficult to extend his patches to the PBA, but unfortunately neither him (neither us) don't have this specific hardware.

                          The patches he made are actually upstream b2cd07a0447bfa25e96ae13e190225b61a3670cb so you can take a look at it if you want.

                          I will try to see if we have an easy possibility to get this HW

                          1 Reply Last reply Reply Quote 0
                          • olivierlambertO Offline
                            olivierlambert Vates 🪐 Co-Founder CEO
                            last edited by

                            The other way is to build a version that "should work" and make it test by the community 🙂

                            1 Reply Last reply Reply Quote 0
                            • andSmvA Offline
                              andSmv Vates 🪐 XCP-ng Team Xen Guru @milch
                              last edited by

                              @milch I will take a look this week and try to figure out if we can make progress on that, so you could have something to test.

                              N 1 Reply Last reply Reply Quote 1
                              • N Offline
                                Niall Con @andSmv
                                last edited by

                                @andSmv @olivierlambert
                                Hi Guys,
                                I'm happy to help. I wouldn't say I'm overly technical but not a noob either.
                                I have a clean install with the hardware. I can arrange a remote connection to the machine if that makes it easier.

                                Let me know and I'll do my best to assist.

                                andSmvA 1 Reply Last reply Reply Quote 1
                                • andSmvA Offline
                                  andSmv Vates 🪐 XCP-ng Team Xen Guru @Niall Con
                                  last edited by

                                  @Niall-Con Thank you! I'll take a look to that and will ping you to test on real hardware. Just need to find time (in the middle of storm right now), so it'll take one or two weeks most probably.

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post