XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Coral TPU PCI Passthrough

    Scheduled Pinned Locked Moved Compute
    38 Posts 7 Posters 7.8k Views 11 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R Offline
      redakula @andSmv
      last edited by

      @andSmv
      As expected the VM with the coral m2 crashes on boot.

      Where would i start with building a custom Xen? The Koji docs seem directed at authorized package maintainers so would i need to build the sources directly from Xen?
      Feeling old admitting it was in the 2.6 days i last regularly built custom kernels 😊

      andSmvA 1 Reply Last reply Reply Quote 0
      • andSmvA Offline
        andSmv Vates 🪐 XCP-ng Team Xen Guru @redakula
        last edited by

        @redakula I'm on it. I keep you posted.

        andSmvA 1 Reply Last reply Reply Quote 1
        • andSmvA Offline
          andSmv Vates 🪐 XCP-ng Team Xen Guru @andSmv
          last edited by andSmv

          @andSmv
          Hello,
          I integrated Marek's patch and builded a rpm, so you can install (may be need to force rpm install or extract the xen.gz from rpm and install it manually if you prefer)

          Obviously there's no guarantee, it'll work in your case. Moreover, I didn't test the patch, so please backup all your data. It should be harmless, but....

          Here's the link you can download the rpm (should be operational until the end of the month) https://nextcloud.vates.fr/index.php/s/gd7kMwxHtNEP329

          Don't hesitate to ping me if you experience any issue to download/install/... the patched xen.

          Hope it helps!

          P.S. Be sure you're running 8.3 XCP-ng, as I only uploaded xen hypervisor rpm (and not libs/tools which come within)

          R 1 Reply Last reply Reply Quote 0
          • R Offline
            redakula @andSmv
            last edited by

            @andSmv Thanks! 👍 😄

            I tried to be as uninvasive as possible and changed the symbolic link xen.gz to point to the xen.gz from the RPM you created.

            Unfortunately still the same error (It does seem to boot the xen from the RPM as this has version 4.17.3-3 vs. the one currently in the repos which has version 4.17.3-4).

            [2024-05-24 17:06:33] (XEN) [  674.051176] Domain 14 (vcpu#2) crashed on cpu#22:
[2024-05-24 17:06:33] (XEN) [  674.051178] ----[ Xen-4.17.3-3  x86_64  debug=n  Not tainted ]----
[2024-05-24 17:06:33] (XEN) [  674.051179] CPU:    22
[2024-05-24 17:06:33] (XEN) [  674.051180] RIP:    0010:[<ffffffffa8581584>]
[2024-05-24 17:06:33] (XEN) [  674.051180] RFLAGS: 0000000000000286   CONTEXT: hvm guest (d14v2)
[2024-05-24 17:06:33] (XEN) [  674.051182] rax: ffffbd9c00149800   rbx: ffff9e9247cc9000   rcx: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051182] rdx: 00000000fee77000   rsi: 0000000000000000   rdi: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051183] rbp: ffffbd9c00327690   rsp: ffffbd9c00327658   r8:  0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051183] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051184] r12: ffffbd9c003276ac   r13: 0000000000000011   r14: ffff9e92413390c0
[2024-05-24 17:06:33] (XEN) [  674.051185] r15: 0000000000000077   cr0: 0000000080050033   cr4: 0000000000750ef0
[2024-05-24 17:06:33] (XEN) [  674.051185] cr3: 0000000103806000   cr2: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051186] fsb: 00007b6e7a42a8c0   gsb: ffff9e925b500000   gss: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051186] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0018   cs: 0010

            It does appear that there is some movement upstream on this (if i interpret the xen mailing list correctly).
            This patch series references the same title as the patch in this thread from 2022 and a bunch of other related work:
            https://lore.kernel.org/xen-devel/cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com/

            andSmvA 1 Reply Last reply Reply Quote 0
            • andSmvA Offline
              andSmv Vates 🪐 XCP-ng Team Xen Guru @redakula
              last edited by

              @redakula
              Well, this was unfortunately one of the potential outcome. Unfortunately we don't have the hardware to make more "in deep" debug. I will talk to Marek next week (on Xen Summit) about this patch series and if we could expect it eventually fix the issue with Coral TPU.
              Will keep you posted.

              R 1 Reply Last reply Reply Quote 2
              • R Offline
                redakula @andSmv
                last edited by

                @andSmv

                Thanks 🙂
                Let me know and i will be happy to continue testing 👍

                1 Reply Last reply Reply Quote 0
                • R redakula referenced this topic on
                • N Offline
                  Niall Con
                  last edited by

                  This post is deleted!
                  1 Reply Last reply Reply Quote 0
                  • N Offline
                    Niall Con
                    last edited by

                    Hi.
                    I'm just wondering if anyone ever got this to work? It's been a long time since I trialled xcp-ng. I really wanted to use it use the coral pci tpu for a Frigate NVR solution.
                    The coral.ai website only provides driver installation instructions for debian based systems.

                    I purchased a dual edge M.2 version and put it in the M.2 that was being used by the Wifi card (which I don't need.
                    This went into a mini pc running a Ryzen 5800u.

                    When I lspci -nn | grep 089a looking to see if the host machine recognises it. I get the following:

                    03:00.0 Non-VGA unclassified device [0000]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

                    So it does see it. But when I look in Xen Orchestra, there's no sign.

                    Was anyone successful in getting this to work?

                    Thank you.

                    R 1 Reply Last reply Reply Quote 0
                    • R Offline
                      redakula @Niall Con
                      last edited by

                      @Niall-Con

                      Nope - i tested recently with the latest updates and the vm still crashes...

                      There were some patches for Xen and i tested the one @andSmv posted without any change.
                      There looked to be quite a few patches around similar issues on upstream Xen but i have no idea if any of them were relevant.

                      Maybe we will see improvement once Xcp-ng 9 comes about with a more recent version?

                      N 1 Reply Last reply Reply Quote 0
                      • olivierlambertO Offline
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        Ping @andSmv I don't remember the status of this

                        1 Reply Last reply Reply Quote 0
                        • N Offline
                          Niall Con @redakula
                          last edited by

                          @redakula Thanks for letting me know.
                          I guess I'll just have to go with a dedicated low power stand alone device for Frigate and keep an eye out in the future to see if it makes it's way into Xcp-ng 9.

                          1 Reply Last reply Reply Quote 0
                          • andSmvA Offline
                            andSmv Vates 🪐 XCP-ng Team Xen Guru
                            last edited by

                            I don't aware if there's something new from Marek who initially worked on these patches. I think at the time he addressed not this particular hardware but rather the global issue, and this patch wasn't tested with Coral HW, so most probably that's why it doesn't work (may be more issues...)

                            I will ping Marek on XEN Community Matrix channel to know if there's something new at that level and will keep you posted.

                            M 1 Reply Last reply Reply Quote 1
                            • M Offline
                              milch @andSmv
                              last edited by

                              @andSmv Did you ever hear back by any chance?

                              andSmvA 2 Replies Last reply Reply Quote 0
                              • andSmvA Offline
                                andSmv Vates 🪐 XCP-ng Team Xen Guru @milch
                                last edited by

                                Just got the answer from Marek on that. The patches he made was tested with Intel Wifi cards and was targeting the similar issue (MSI-x table) but not the same as Coral TPU (PBA). It should be not very very difficult to extend his patches to the PBA, but unfortunately neither him (neither us) don't have this specific hardware.

                                The patches he made are actually upstream b2cd07a0447bfa25e96ae13e190225b61a3670cb so you can take a look at it if you want.

                                I will try to see if we have an easy possibility to get this HW

                                1 Reply Last reply Reply Quote 0
                                • olivierlambertO Offline
                                  olivierlambert Vates 🪐 Co-Founder CEO
                                  last edited by

                                  The other way is to build a version that "should work" and make it test by the community 🙂

                                  1 Reply Last reply Reply Quote 0
                                  • andSmvA Offline
                                    andSmv Vates 🪐 XCP-ng Team Xen Guru @milch
                                    last edited by

                                    @milch I will take a look this week and try to figure out if we can make progress on that, so you could have something to test.

                                    N 1 Reply Last reply Reply Quote 1
                                    • N Offline
                                      Niall Con @andSmv
                                      last edited by

                                      @andSmv @olivierlambert
                                      Hi Guys,
                                      I'm happy to help. I wouldn't say I'm overly technical but not a noob either.
                                      I have a clean install with the hardware. I can arrange a remote connection to the machine if that makes it easier.

                                      Let me know and I'll do my best to assist.

                                      andSmvA 1 Reply Last reply Reply Quote 1
                                      • andSmvA Offline
                                        andSmv Vates 🪐 XCP-ng Team Xen Guru @Niall Con
                                        last edited by

                                        @Niall-Con Thank you! I'll take a look to that and will ping you to test on real hardware. Just need to find time (in the middle of storm right now), so it'll take one or two weeks most probably.

                                        1 Reply Last reply Reply Quote 0
                                        • First post
                                          Last post