<![CDATA[Google Coral TPU PCIe Passthrough Woes]]>I recently moved to XCP-ng because I had been unable to get the Google Coral TPU to pass through properly to VMs in ESXi. Unfortunately, passing the Coral through in XCP-ng results in another, different failure, with the guest VM crashing as soon as I install the Google drivers that work fine on bare metal installs of the same Ubuntu 20.04 guest.

The TPU does show up with lspci in the guest, and I've successfully passed through a GPU and a USB controller to different guests.

I'm not sure what the most relevant logs are, but this is what I see in hypervisor.log:

[2022-09-03 15:25:35] (XEN) [ 1373.790117] memory_map: error -22 removing dom11 access to [3fff2100,3fff2103]
[2022-09-03 15:25:35] (XEN) [ 1373.889785] memory_map: error -22 removing dom11 access to [3fff2100,3fff2103]
[2022-09-03 15:25:35] (XEN) [ 1373.988658] memory_map: error -22 removing dom11 access to [3fff2100,3fff2103]
[2022-09-03 15:25:35] (XEN) [ 1374.083144] memory_map: error -22 removing dom11 access to [3fff2100,3fff2103]
[2022-09-03 15:25:35] (XEN) [ 1374.182721] memory_map: error -22 removing dom11 access to [3fff2100,3fff2103]
[2022-09-03 15:25:41] (XEN) [ 1380.209079] d11v0 EPT violation 0x1aa (-w-/r-x) gpa 0x000000f184680c mfn 0x3fff2046 type 5
[2022-09-03 15:25:41] (XEN) [ 1380.209083] d11v0 Walking EPT tables for GFN f1846:
[2022-09-03 15:25:41] (XEN) [ 1380.209086] d11v0  epte 9c000015d1df8107
[2022-09-03 15:25:41] (XEN) [ 1380.209089] d11v0  epte 9c00000e34a20107
[2022-09-03 15:25:41] (XEN) [ 1380.209092] d11v0  epte 9c00000ef3120107
[2022-09-03 15:25:41] (XEN) [ 1380.209094] d11v0  epte 9c5003fff2046945
[2022-09-03 15:25:41] (XEN) [ 1380.209097] d11v0  --- GLA 0xffffb91f001a180c
[2022-09-03 15:25:41] (XEN) [ 1380.209107] domain_crash called from vmx_vmexit_handler+0xf55/0x19c0
[2022-09-03 15:25:41] (XEN) [ 1380.209110] Domain 11 (vcpu#0) crashed on cpu#39:
[2022-09-03 15:25:41] (XEN) [ 1380.209116] ----[ Xen-4.13.4-9.24.1  x86_64  debug=n   Not tainted ]----
[2022-09-03 15:25:41] (XEN) [ 1380.209119] CPU:    39
[2022-09-03 15:25:41] (XEN) [ 1380.209122] RIP:    0010:[<ffffffff9ff8ccdd>]
[2022-09-03 15:25:41] (XEN) [ 1380.209124] RFLAGS: 0000000000010246   CONTEXT: hvm guest (d11v0)
[2022-09-03 15:25:41] (XEN) [ 1380.209129] rax: 0000000000000000   rbx: ffffb91f001a1800   rcx: 0000000000000080
[2022-09-03 15:25:41] (XEN) [ 1380.209132] rdx: ffffb91f001a180c   rsi: 0000000000000001   rdi: 0000000000000000
[2022-09-03 15:25:41] (XEN) [ 1380.209135] rbp: ffffb91f0059f968   rsp: ffffb91f0059f8f0   r8:  0000000000000000
[2022-09-03 15:25:41] (XEN) [ 1380.209139] r9:  ffffb91f0059f7a8   r10: ffffb91f00000000   r11: ffffa059148c2f40
[2022-09-03 15:25:41] (XEN) [ 1380.209142] r12: 0000000000000000   r13: ffffa05916b35000   r14: ffffa0590c723080
[2022-09-03 15:25:41] (XEN) [ 1380.209145] r15: 000000000000000d   cr0: 0000000080050033   cr4: 00000000001606f0
[2022-09-03 15:25:41] (XEN) [ 1380.209147] cr3: 00000001945b2001   cr2: 000055c9503250c0
[2022-09-03 15:25:41] (XEN) [ 1380.209150] fsb: 00007feabbd8d880   gsb: ffffa05918400000   gss: 0000000000000000
[2022-09-03 15:25:41] (XEN) [ 1380.209153] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0018   cs: 0010
]]>https://xcp-ng.org/forum/topic/6304/google-coral-tpu-pcie-passthrough-woesRSS for NodeWed, 13 May 2026 01:28:33 GMTSat, 10 Sep 2022 15:17:43 GMT60<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Wed, 13 Nov 2024 00:19:03 GMT]]>Passing through the usb adapter I am getting roughly 30ms response times which is acceptable. Definitely not as fast as the m.2 version but good enough to keep me from buying a mini-pc or SBC. I would love to switch to the m.2 if someone could post here if they get a success. Just a strange side note, I couldn't get it to list in xoa and use the gui to do the passthrough. I had to manually hide it from the dom and pass the usb adapter to the vm. I couldn't figure out how to refresh the pci list and tried doing reboots to see if it would pick up the new device but no luck. If anyone knows how to refresh that list I'd be interested.

Thanks,
SFD

]]>https://xcp-ng.org/forum/post/85687https://xcp-ng.org/forum/post/85687Wed, 13 Nov 2024 00:19:03 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Mon, 11 Nov 2024 15:49:35 GMT]]>Having the same issue, tried all different kernels and patches with no luck. Found this thread here and have some hope. I have both the m.2 and usb versions of the coral. So I have a pcie usb 3.2 adapter card coming tomorrow and I am going to try to pass the card through instead of the m.2 and hook up the usb version see if I can get some performance. I'll let ya guys know if it is acceptable speed.

]]>https://xcp-ng.org/forum/post/85611https://xcp-ng.org/forum/post/85611Mon, 11 Nov 2024 15:49:35 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Wed, 06 Nov 2024 16:28:32 GMT]]>@Teddy-Astie

I think that is the patch i tested here:
https://xcp-ng.org/forum/topic/7066/coral-tpu-pci-passthrough/26?_=1730909872550

And no it made no difference...
I don't know if @andSmv has any more info? - ref. the thread above.

There seems to be a lot of work in the Xen repo on MSI stuff but i could not figure out what would be relevant for the Coral.

]]>https://xcp-ng.org/forum/post/85444https://xcp-ng.org/forum/post/85444Wed, 06 Nov 2024 16:28:32 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Wed, 06 Nov 2024 14:42:26 GMT]]>@slavox The patch I linked is not applied to current XCP-ng.
But even if it was, it would still not work due to the MSI-X/PBA/registers issue in a same page I quoted previously.
It's not a simple issue to tackle on, but upstream Xen is aware of that and it may be solved in the future (difficult to put an ETA though).

]]>https://xcp-ng.org/forum/post/85435https://xcp-ng.org/forum/post/85435Wed, 06 Nov 2024 14:42:26 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Tue, 05 Nov 2024 19:48:55 GMT]]>@Teddy-Astie Is this patch already in the current kernel or do i need to manually apply it?

]]>https://xcp-ng.org/forum/post/85393https://xcp-ng.org/forum/post/85393Tue, 05 Nov 2024 19:48:55 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Tue, 05 Nov 2024 12:51:56 GMT]]>I think it is the same MSI-X/PBA issues that may be partially fixed with https://gitlab.com/xen-project/xen/-/commit/b2cd07a0447bfa25e96ae13e190225b61a3670cb

However, with this device, MSI-X vector table and PBA are in a same page (vector table in 46800 and PBA in 46068) though, which is threated a bit differently

If PBA lives on the same page, discard writes and log a message.
Technically, writes outside of PBA could be allowed, but at this moment
the precise location of PBA isn't saved, and also no known device abuses
the spec in this way (at least yet).

But Coral appears to abuse this according to DKMS driver by having more than just MSI-X and PBA on a single page
https://github.com/google/gasket-driver/blob/main/src/apex_driver.c#L103-L140

]]>https://xcp-ng.org/forum/post/85370https://xcp-ng.org/forum/post/85370Tue, 05 Nov 2024 12:51:56 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Tue, 05 Nov 2024 07:00:19 GMT]]>@Teddy-Astie if you have some bandwidth, can you take a look?

]]>https://xcp-ng.org/forum/post/85357https://xcp-ng.org/forum/post/85357Tue, 05 Nov 2024 07:00:19 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Tue, 05 Nov 2024 02:39:16 GMT]]>hey @andSmv @olivierlambert
I have a PCI coral TPU and have the same issue from this thread. It doesn't look like anyone confirmed if the patch is working.

Anything I can do to help test here? I have just switched away from proxmox so would prefer to get it working in XCP
I'm currently on 8.3 and the alt kernel. But happy to test with whatever, I have some spare hardware to setup a dedicated test if needed.

uname -a
Linux xcp-long 4.19.316+1 #1 SMP Mon Aug 19 14:31:42 CEST 2024 x86_64 x86_64 x86_64 GNU/Linux

xl dmesg

(XEN) [ 3010.009205] d12v5 EPT violation 0x1aa (-w-/r-x) gpa 0x000000f1846800 mfn 0x90246 type 5
(XEN) [ 3010.009207] d12v5 Walking EPT tables for GFN f1846:
(XEN) [ 3010.009209] d12v5  epte 9c00000cb3924007
(XEN) [ 3010.009210] d12v5  epte 9c0000084c552007
(XEN) [ 3010.009211] d12v5  epte 9c00000847e9d007
(XEN) [ 3010.009212] d12v5  epte 9c50000090246845
(XEN) [ 3010.009214] d12v5  --- GLA 0xffffaea6c0d8d800
(XEN) [ 3010.009219] domain_crash called from vmx_vmexit_handler+0xa8d/0x1ab0
(XEN) [ 3010.009221] Domain 12 (vcpu#5) crashed on cpu#17:
(XEN) [ 3010.009225] ----[ Xen-4.17.5-3  x86_64  debug=n  Not tainted ]----
(XEN) [ 3010.009226] CPU:    17
(XEN) [ 3010.009227] RIP:    0010:[<ffffffff8dd86326>]
(XEN) [ 3010.009228] RFLAGS: 0000000000010286   CONTEXT: hvm guest (d12v5)
(XEN) [ 3010.009231] rax: ffffaea6c0d8d800   rbx: ffff88c634a53800   rcx: 0000000000000000
(XEN) [ 3010.009232] rdx: 00000000fee87000   rsi: 0000000000000000   rdi: 0000000000000000
(XEN) [ 3010.009234] rbp: ffffaea6c0b0f448   rsp: ffffaea6c0b0f410   r8:  0000000000000000
(XEN) [ 3010.009235] r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) [ 3010.009236] r12: ffffaea6c0b0f464   r13: 0000000000000011   r14: ffff88c6022860c8
(XEN) [ 3010.009238] r15: 0000000000000087   cr0: 0000000080050033   cr4: 00000000001006f0
(XEN) [ 3010.009239] cr3: 0000000105aca000   cr2: 00007b3046869000
(XEN) [ 3010.009240] fsb: 000079ea9326d8c0   gsb: ffff88cb07280000   gss: 0000000000000000
(XEN) [ 3010.009242] ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0018   cs: 0010

lspci -vvv -s

lspci -vvv -s 86:00.0
86:00.0 System peripheral: Global Unichip Corp. Coral Edge TPU (prog-if ff)
	Subsystem: Global Unichip Corp. Coral Edge TPU
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 56
	Region 0: Memory at 901fc000 (64-bit, prefetchable) [size=16K]
	Region 2: Memory at 90200000 (64-bit, prefetchable) [size=1M]
	Capabilities: [80] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal+ Fatal+ Unsupported-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 256 bytes, MaxReadReq 4096 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
			ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported
		DevCtl2: Completion Timeout: 260ms to 900ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [d0] MSI-X: Enable- Count=128 Masked-
		Vector table: BAR=2 offset=00046800
		PBA: BAR=2 offset=00046068
	Capabilities: [e0] MSI: Enable- Count=1/32 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [f8] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Vendor Specific Information: ID=1556 Rev=1 Len=008 <?>
	Capabilities: [108 v1] Latency Tolerance Reporting
		Max snoop latency: 0ns
		Max no snoop latency: 0ns
	Capabilities: [110 v1] L1 PM Substates
		L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
			  PortCommonModeRestoreTime=10us PortTPowerOnTime=10us
	Capabilities: [200 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP+ CmpltTO+ CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC+ UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Kernel driver in use: pciback
]]>https://xcp-ng.org/forum/post/85351https://xcp-ng.org/forum/post/85351Tue, 05 Nov 2024 02:39:16 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Sun, 18 Aug 2024 12:51:07 GMT]]>@olivierlambert Thanks, don't worry in that case, was just to see if there was something like "oh yeah XCP does [something] with vUSBs when passing through which could explain it". The server is a mini PC so no PCIe card slots or capability unfortunately.

I'll just live with 40ms via VirtualHere (don't know why that's so high either as others have 15-20 with that method)! It works well enough.

]]>https://xcp-ng.org/forum/post/81659https://xcp-ng.org/forum/post/81659Sun, 18 Aug 2024 12:51:07 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Sun, 18 Aug 2024 12:46:02 GMT]]>Hi,

I don't know internal mechanism of the vUSB thing and why it cause this on your device (which is really a special device, with its own quirks).

I don't remember if you already tried to passthrough a PCIe USB adapter card, then plug the USB device on it and see if it's better than vUSB?

]]>https://xcp-ng.org/forum/post/81657https://xcp-ng.org/forum/post/81657Sun, 18 Aug 2024 12:46:02 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Sun, 18 Aug 2024 12:41:16 GMT]]>So I eventually got round to trying the USB Coral via passthrough, which worked great, but the TPU itself exhibited some behavior that made it nonviable which sucks. The USB was actually detected by XO as Google Inc. and Frigate actually loaded the TPU, but the inference speed was in excess of 180ms (it should be around 10, USB over IP it's 40). So it worked but, didn't.

The normal procedure with a Coral is to run a make reset from their utilities which switches the TPU back to runtime mode. This worked under my current (and now reverted) system of VirtualHere USB over IP, but it didn't work when passed through.

Output of make reset:

dfu-util: Warning: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release
dfu-util: No DFU capable USB device available

It should look like this:

Opening DFU capable USB device...
Device ID 1a6e:089a
Device DFU version 0101
Claiming USB DFU Interface...
Setting Alternate Interface #0 ...
Determining device status...
DFU state(2) = dfuIDLE, status(0) = No error condition is present
DFU mode device DFU version 0101
Device returned transfer size 256
Copying data from PC to DFU device
Download	[=========================] 100%        10783 bytes
Download done.
DFU state(2) = dfuIDLE, status(0) = No error condition is present
Done!
Resetting USB to switch back to Run-Time mode

Sorry to ping you @olivierlambert but would you happen to know what might cause this in XCP/XO? Is there something going on when the device is made into a vUSB that would cause it to error out/be inaccessible in DFU (I assume this means Device Firmware Update)?

]]>https://xcp-ng.org/forum/post/81656https://xcp-ng.org/forum/post/81656Sun, 18 Aug 2024 12:41:16 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Sat, 17 Feb 2024 19:49:19 GMT]]>@olivierlambert Thanks that's good to know. That functionality would be great down the line!

I do have a spare M.2 E-key on my XCP host running the VM Coral is needed for, but seems like I'd have trouble going by this thread. Might even have trouble with the USB Coral, it hasn't been much better so far in terms of whacky non-standard behavior...

]]>https://xcp-ng.org/forum/post/71544https://xcp-ng.org/forum/post/71544Sat, 17 Feb 2024 19:49:19 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Sat, 17 Feb 2024 16:49:14 GMT]]>For NOBAK and on 8.3 yes, but I'm not sure it will be related to USB. You should use offline, that should work. Alternatively, we have plans to detect the error, to unplug the USB device, do the snap and replug it just after.

]]>https://xcp-ng.org/forum/post/71538https://xcp-ng.org/forum/post/71538Sat, 17 Feb 2024 16:49:14 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Sat, 17 Feb 2024 16:06:04 GMT]]>@olivierlambert Seems like a reasonable place to ask as any - I am currently using a USB Coral over IP (Virtualhere) but would rather load it into my VM directly - what's the current status of snapshots/backups with a vUSB?

I've been reading that XO can now support disk exclusions with [NOBAK] but this probably doesn't apply to a Coral. Is an offline backup still the best available method?

]]>https://xcp-ng.org/forum/post/71533https://xcp-ng.org/forum/post/71533Sat, 17 Feb 2024 16:06:04 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Tue, 23 Jan 2024 21:47:44 GMT]]>PCI passthrough might cause problems with this device, but USB could work.

]]>https://xcp-ng.org/forum/post/70018https://xcp-ng.org/forum/post/70018Tue, 23 Jan 2024 21:47:44 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Tue, 23 Jan 2024 21:34:08 GMT]]>@Nornode hey, nope I did not. I ended up moving my infrastructure to Proxmox.

Honestly this is no fault of XCP-ng and XCP-ng suits my hardware / setup a lot better, but it was either that or I had two servers that needed to be bare metal.

]]>https://xcp-ng.org/forum/post/70017https://xcp-ng.org/forum/post/70017Tue, 23 Jan 2024 21:34:08 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Tue, 23 Jan 2024 20:09:37 GMT]]>@jmccoy555 // @jjgg

Did anyone of you get your Coral USB TPU working and passthrough to a VM?

]]>https://xcp-ng.org/forum/post/70016https://xcp-ng.org/forum/post/70016Tue, 23 Jan 2024 20:09:37 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Sat, 13 May 2023 10:23:03 GMT]]>@jjgg it would be great if we could get this working. My CPU utilisation is fine too, but when I shut down my Zoneminder VM things go a lot quieter (fans) so I'm sure there would be a benefit CPU and power wise.

]]>https://xcp-ng.org/forum/post/62188https://xcp-ng.org/forum/post/62188Sat, 13 May 2023 10:23:03 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Fri, 12 May 2023 22:02:37 GMT]]>Definitely frustrating and no fault of xcp-ng - I have a lot of spare cpu cycles so it isn't majorly impacting me that I know of. I'm still available to test fixes though.

Looks like most of the Proxmox users have got this working in an LXC container by installing the drivers on the host itself and passing through the actual Apex devices. Not a route that's applicable to us but just a datapoint.

]]>https://xcp-ng.org/forum/post/62180https://xcp-ng.org/forum/post/62180Fri, 12 May 2023 22:02:37 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Fri, 12 May 2023 20:51:07 GMT]]>Maybe this one will come to life again https://xcp-ng.org/forum/topic/7066/coral-tpu-pci-passthrough/14

Don't really want to buy one knowing its not working!!

]]>https://xcp-ng.org/forum/post/62175https://xcp-ng.org/forum/post/62175Fri, 12 May 2023 20:51:07 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Fri, 12 May 2023 20:40:58 GMT]]>If only they could have done PCI hardware that follow the PCI specifications 😢

]]>https://xcp-ng.org/forum/post/62174https://xcp-ng.org/forum/post/62174Fri, 12 May 2023 20:40:58 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Fri, 12 May 2023 20:39:56 GMT]]>Ah, just found these things exist..... Then just found this issue exists too ☹️

]]>https://xcp-ng.org/forum/post/62173https://xcp-ng.org/forum/post/62173Fri, 12 May 2023 20:39:56 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Mon, 24 Apr 2023 02:27:54 GMT]]>Booted into fallback and put things back the way they were. Happy to keep testing if there's additional bits to test.

]]>https://xcp-ng.org/forum/post/61355https://xcp-ng.org/forum/post/61355Mon, 24 Apr 2023 02:27:54 GMT<![CDATA[Reply to Google Coral TPU PCIe Passthrough Woes on Mon, 24 Apr 2023 02:14:02 GMT]]>Ok so testing setup.

Downloaded xen.gz, renamed to xenept.gz and put into /boot:

df0800ca-42eb-4675-a4cd-f0afc93ab3fd-image.png

Updated grub.cfg to point to that file:

1d2216bc-7194-4acb-8607-1513b348f6ca-image.png

Rebooted host.

Got to the grub screen, let it load as normal, as soon as it disappeared (so it had made the default selection) the server rebooted.

I ended up just placing xen.gz in /boot and removing the symbolic link to the original and attempting, no difference.

Of note, messing with kernels / grub is not something I've got experience with. I may have made a mistake / need things explained in a bit more detail if I'm potentially misunderstood some instructions above.

]]>https://xcp-ng.org/forum/post/61354https://xcp-ng.org/forum/post/61354Mon, 24 Apr 2023 02:14:02 GMT