<![CDATA[Coral TPU PCI Passthrough]]>Anyone ever get PCI passthrough to work for Coral TPU?
After enabling the passthrough the Debian 11 VM won't start and the log shows:
unix enoent open error.

[12:48 xcp-lab0 ~]# lspci -vvv -s 04:00.0
04:00.0 Non-VGA unclassified device: Global Unichip Corp. Coral Edge TPU (prog-if ff)
        Subsystem: Global Unichip Corp. Coral Edge TPU
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 255
        Region 0: Memory at 4000100000 (64-bit, prefetchable) [disabled] [size=16K]
        Region 2: Memory at 4000000000 (64-bit, prefetchable) [disabled] [size=1M]
        Capabilities: [80] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 25.000W
                DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
                        RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
                LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                        ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled
                LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
        Capabilities: [d0] MSI-X: Enable- Count=128 Masked-
                Vector table: BAR=2 offset=00046800
                PBA: BAR=2 offset=00046068
        Capabilities: [e0] MSI: Enable- Count=1/32 Maskable- 64bit+
                Address: 0000000000000000  Data: 0000
        Capabilities: [f8] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [100 v1] Vendor Specific Information: ID=1556 Rev=1 Len=008 <?>
        Capabilities: [108 v1] Latency Tolerance Reporting
                Max snoop latency: 3145728ns
                Max no snoop latency: 3145728ns
        Capabilities: [110 v1] L1 PM Substates
                L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
                          PortCommonModeRestoreTime=10us PortTPowerOnTime=10us
        Capabilities: [200 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-

[12:48 xcp-lab0 ~]#
]]>https://xcp-ng.org/forum/topic/7066/coral-tpu-pci-passthroughRSS for NodeFri, 17 Apr 2026 03:43:30 GMTThu, 09 Mar 2023 20:57:01 GMT60<![CDATA[Reply to Coral TPU PCI Passthrough on Fri, 20 Feb 2026 21:53:46 GMT]]>@redakula said in Coral TPU PCI Passthrough:

Frigate.nvr which is one of the popular uses for the Coral do not recommend it for new installs either.

Frigate updated their recommendations because of the Google decision to sunset the device and because there are alternative options available to frigate for image inferencing. The coral is still supported though, and frigate is not the only use case or platform that can benefit from an accelerator.

At the end of the day if you've already got the hardware, and it's efficient enough to run, then not using it is a waste or resources that could be allocated to other VMs.

]]>https://xcp-ng.org/forum/post/102782https://xcp-ng.org/forum/post/102782Fri, 20 Feb 2026 21:53:46 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Wed, 07 Jan 2026 17:15:36 GMT]]>@andSmv
I definitely think it would be nice with support even though i repurposed my M2 Coral for other uses.
But one thing to keep in mind regarding resources is that the product would appear to be heading to https://killedbygoogle.com
coral.ai redirects to their new NPU offering and no updates have been made from google to the driver since 2022... Mouser still has plenty of stock though.

Frigate.nvr which is one of the popular uses for the Coral do not recommend it for new installs either.

]]>https://xcp-ng.org/forum/post/101354https://xcp-ng.org/forum/post/101354Wed, 07 Jan 2026 17:15:36 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Wed, 07 Jan 2026 15:22:41 GMT]]>Sorry guys. This one is harder than expected, basically the marek patch addresses only MSI-X Vector Table (and which has some emulation support included in Xen). MSI-x PBA related stuff is a different - so I will try to see how I can get the actual device to fix this one

]]>https://xcp-ng.org/forum/post/101349https://xcp-ng.org/forum/post/101349Wed, 07 Jan 2026 15:22:41 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Tue, 16 Dec 2025 10:16:35 GMT]]>@firereverie Sorry, these last few weeks was very-very busy. So I REALLY hope will do something on this before the december end and come back to you.

]]>https://xcp-ng.org/forum/post/100707https://xcp-ng.org/forum/post/100707Tue, 16 Dec 2025 10:16:35 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 15 Dec 2025 20:20:08 GMT]]>@andSmv @olivierlambert Mid-December circle back.

Anything promising for us to test before the new year?

If not; happy non-denominational holiday and new year.

]]>https://xcp-ng.org/forum/post/100693https://xcp-ng.org/forum/post/100693Mon, 15 Dec 2025 20:20:08 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 03 Nov 2025 16:29:51 GMT]]>Hey, sorry guys. Didn't have time to advance on that. I promise will put something to test during november.

]]>https://xcp-ng.org/forum/post/99189https://xcp-ng.org/forum/post/99189Mon, 03 Nov 2025 16:29:51 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 03 Nov 2025 08:24:34 GMT]]>Reping @andSmv

]]>https://xcp-ng.org/forum/post/99159https://xcp-ng.org/forum/post/99159Mon, 03 Nov 2025 08:24:34 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Sun, 02 Nov 2025 22:05:35 GMT]]>Also ended up here while researching/troubleshooting this hardware; Has there been any additional movement on this topic?

]]>https://xcp-ng.org/forum/post/99149https://xcp-ng.org/forum/post/99149Sun, 02 Nov 2025 22:05:35 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Fri, 10 Oct 2025 20:44:47 GMT]]>@andSmv Any news on a test build with the patch? I'm wondering if this issue is related and would love to be able to test.

]]>https://xcp-ng.org/forum/post/98356https://xcp-ng.org/forum/post/98356Fri, 10 Oct 2025 20:44:47 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Wed, 02 Jul 2025 12:49:02 GMT]]>@Niall-Con Thank you! I'll take a look to that and will ping you to test on real hardware. Just need to find time (in the middle of storm right now), so it'll take one or two weeks most probably.

]]>https://xcp-ng.org/forum/post/94747https://xcp-ng.org/forum/post/94747Wed, 02 Jul 2025 12:49:02 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Sat, 28 Jun 2025 10:08:01 GMT]]>@andSmv @olivierlambert
Hi Guys,
I'm happy to help. I wouldn't say I'm overly technical but not a noob either.
I have a clean install with the hardware. I can arrange a remote connection to the machine if that makes it easier.

Let me know and I'll do my best to assist.

]]>https://xcp-ng.org/forum/post/94519https://xcp-ng.org/forum/post/94519Sat, 28 Jun 2025 10:08:01 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 23 Jun 2025 14:18:11 GMT]]>@milch I will take a look this week and try to figure out if we can make progress on that, so you could have something to test.

]]>https://xcp-ng.org/forum/post/94264https://xcp-ng.org/forum/post/94264Mon, 23 Jun 2025 14:18:11 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 23 Jun 2025 13:47:40 GMT]]>The other way is to build a version that "should work" and make it test by the community 🙂

]]>https://xcp-ng.org/forum/post/94255https://xcp-ng.org/forum/post/94255Mon, 23 Jun 2025 13:47:40 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 23 Jun 2025 13:40:34 GMT]]>Just got the answer from Marek on that. The patches he made was tested with Intel Wifi cards and was targeting the similar issue (MSI-x table) but not the same as Coral TPU (PBA). It should be not very very difficult to extend his patches to the PBA, but unfortunately neither him (neither us) don't have this specific hardware.

The patches he made are actually upstream b2cd07a0447bfa25e96ae13e190225b61a3670cb so you can take a look at it if you want.

I will try to see if we have an easy possibility to get this HW

]]>https://xcp-ng.org/forum/post/94254https://xcp-ng.org/forum/post/94254Mon, 23 Jun 2025 13:40:34 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 23 Jun 2025 03:31:54 GMT]]>@andSmv Did you ever hear back by any chance?

]]>https://xcp-ng.org/forum/post/94216https://xcp-ng.org/forum/post/94216Mon, 23 Jun 2025 03:31:54 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Wed, 18 Jun 2025 12:58:28 GMT]]>I don't aware if there's something new from Marek who initially worked on these patches. I think at the time he addressed not this particular hardware but rather the global issue, and this patch wasn't tested with Coral HW, so most probably that's why it doesn't work (may be more issues...)

I will ping Marek on XEN Community Matrix channel to know if there's something new at that level and will keep you posted.

]]>https://xcp-ng.org/forum/post/94041https://xcp-ng.org/forum/post/94041Wed, 18 Jun 2025 12:58:28 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Wed, 18 Jun 2025 10:08:53 GMT]]>@redakula Thanks for letting me know.
I guess I'll just have to go with a dedicated low power stand alone device for Frigate and keep an eye out in the future to see if it makes it's way into Xcp-ng 9.

]]>https://xcp-ng.org/forum/post/94025https://xcp-ng.org/forum/post/94025Wed, 18 Jun 2025 10:08:53 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Sun, 15 Jun 2025 13:04:18 GMT]]>Ping @andSmv I don't remember the status of this

]]>https://xcp-ng.org/forum/post/93949https://xcp-ng.org/forum/post/93949Sun, 15 Jun 2025 13:04:18 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Sun, 15 Jun 2025 07:31:33 GMT]]>@Niall-Con

Nope - i tested recently with the latest updates and the vm still crashes...

There were some patches for Xen and i tested the one @andSmv posted without any change.
There looked to be quite a few patches around similar issues on upstream Xen but i have no idea if any of them were relevant.

Maybe we will see improvement once Xcp-ng 9 comes about with a more recent version?

]]>https://xcp-ng.org/forum/post/93944https://xcp-ng.org/forum/post/93944Sun, 15 Jun 2025 07:31:33 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Sat, 14 Jun 2025 17:11:55 GMT]]>Hi.
I'm just wondering if anyone ever got this to work? It's been a long time since I trialled xcp-ng. I really wanted to use it use the coral pci tpu for a Frigate NVR solution.
The coral.ai website only provides driver installation instructions for debian based systems.

I purchased a dual edge M.2 version and put it in the M.2 that was being used by the Wifi card (which I don't need.
This went into a mini pc running a Ryzen 5800u.

When I lspci -nn | grep 089a looking to see if the host machine recognises it. I get the following:

03:00.0 Non-VGA unclassified device [0000]: Global Unichip Corp. Coral Edge TPU [1ac1:089a]

So it does see it. But when I look in Xen Orchestra, there's no sign.

Was anyone successful in getting this to work?

Thank you.

]]>https://xcp-ng.org/forum/post/93935https://xcp-ng.org/forum/post/93935Sat, 14 Jun 2025 17:11:55 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 27 May 2024 15:16:50 GMT]]>@andSmv

Thanks 🙂
Let me know and i will be happy to continue testing 👍

]]>https://xcp-ng.org/forum/post/77979https://xcp-ng.org/forum/post/77979Mon, 27 May 2024 15:16:50 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Mon, 27 May 2024 12:39:47 GMT]]>@redakula
Well, this was unfortunately one of the potential outcome. Unfortunately we don't have the hardware to make more "in deep" debug. I will talk to Marek next week (on Xen Summit) about this patch series and if we could expect it eventually fix the issue with Coral TPU.
Will keep you posted.

]]>https://xcp-ng.org/forum/post/77972https://xcp-ng.org/forum/post/77972Mon, 27 May 2024 12:39:47 GMT<![CDATA[Reply to Coral TPU PCI Passthrough on Fri, 24 May 2024 15:24:07 GMT]]>@andSmv Thanks! 👍 😄

I tried to be as uninvasive as possible and changed the symbolic link xen.gz to point to the xen.gz from the RPM you created.

Unfortunately still the same error (It does seem to boot the xen from the RPM as this has version 4.17.3-3 vs. the one currently in the repos which has version 4.17.3-4).

[2024-05-24 17:06:33] (XEN) [  674.051176] Domain 14 (vcpu#2) crashed on cpu#22:
[2024-05-24 17:06:33] (XEN) [  674.051178] ----[ Xen-4.17.3-3  x86_64  debug=n  Not tainted ]----
[2024-05-24 17:06:33] (XEN) [  674.051179] CPU:    22
[2024-05-24 17:06:33] (XEN) [  674.051180] RIP:    0010:[<ffffffffa8581584>]
[2024-05-24 17:06:33] (XEN) [  674.051180] RFLAGS: 0000000000000286   CONTEXT: hvm guest (d14v2)
[2024-05-24 17:06:33] (XEN) [  674.051182] rax: ffffbd9c00149800   rbx: ffff9e9247cc9000   rcx: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051182] rdx: 00000000fee77000   rsi: 0000000000000000   rdi: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051183] rbp: ffffbd9c00327690   rsp: ffffbd9c00327658   r8:  0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051183] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051184] r12: ffffbd9c003276ac   r13: 0000000000000011   r14: ffff9e92413390c0
[2024-05-24 17:06:33] (XEN) [  674.051185] r15: 0000000000000077   cr0: 0000000080050033   cr4: 0000000000750ef0
[2024-05-24 17:06:33] (XEN) [  674.051185] cr3: 0000000103806000   cr2: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051186] fsb: 00007b6e7a42a8c0   gsb: ffff9e925b500000   gss: 0000000000000000
[2024-05-24 17:06:33] (XEN) [  674.051186] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0018   cs: 0010

It does appear that there is some movement upstream on this (if i interpret the xen mailing list correctly).
This patch series references the same title as the patch in this thread from 2022 and a bunch of other related work:
https://lore.kernel.org/xen-devel/cover.33fb4385b7dd6c53bda4acf0a9e91748b3d7b1f7.1715313192.git-series.marmarek@invisiblethingslab.com/

]]>https://xcp-ng.org/forum/post/77871https://xcp-ng.org/forum/post/77871Fri, 24 May 2024 15:24:07 GMT