Hi @robbie-c, could you send a screenshot ? is that in XO ?

XenServer developers recently contributed a patch series that removes a bit of technical debt from Xen, doing which was one of the steps towards proper nested virtualization support. There still remains a large amount of work onwards.

The "PIF is not attached" usually means the network interface selected as the migration network isn't currently active on the target host. It can happen after upgrades if the host hasn't been fully rebooted. Worth checking whether a reboot of both hosts changes anything, and running xe pif-list to see whether that specific PIF shows currently-attached: true on the target. If the PIF looks attached in xe but migration still fails, might be worth a ping to Team-XAPI-Network.

perhaps "in the context of a proceeding RPU, do not start halted VMs" ? or "boot only halted VMs that have HA enabled" ? but I can imagine corner cases where this is not wanted. some chicken & egg problem.

That is a frustrating loop to be in, especially with TrueNAS. Usually, when the VM fails to start after a change, it’s because XCP-ng is trying to pass through a PCI device (like an HBA) that isn't being released properly by the host. Have you checked if the "hide" parameters in your grub config are still correct? Sometimes an update can reset those, and the host grabs the controller before the VM can. Another thing to try is toggling the BIOS/UEFI mode in the VM settings - TrueNAS can be picky about that depending on which version you’re running.

Nevermind. I forgot your previous post about: You'll need to modify two files (via SSH): /opt/dell/omc/utilities/tui/bin/ome_disk_config.sh /opt/dell/mcsi/appliance/scripts/appliance_ressource.sh In these files, replace /dev/sd with /dev/xvd. I went and made these changes, and now the Data drive is loading after doing the drive scan. I should mention that doing this BEFORE first logging into the OME web interface and completing the initialization seems to be best (no further errors appear once logged in).

I can confirm that when using Citrix/Xenserver guest utilities version 8.4 (https://github.com/xenserver/xe-guest-utilities/releases/tag/v8.4.0) memory ballooning / DMC is working fine. After live migration the RAM of the linux guest is expanded to dynamic_max again. So this issue was in fact caused by Rust based xen-guest-agent. For now I'll keep using Citrix/Xenserver guest utilities on my Linux guests until the feature is implemented in Vates rust-based guest utilities. Best regards

@TeddyAstie Attached is the output you requested xen-cpuid -p.txt

@TeddyAstie That's more like it. I'm not finding the com2tcp though. I at least have something to search for. Once configured, is this persistent or do I need to create some sort of start up script that runs/loads a config on machine boot? Thanks!!

@abudef Some quotes from the documentation to clarify the situation: https://docs.xcp-ng.org/compute/#-nested-virtualization

Having help from OpenBSD side could be the way to go. Please forward your report to bugs@openbsd.org (with a link to this post too) For the record, I am able to reproduce it too (on XCP-ng 8.3 too, using OpenBSD -current).

@redakula said in Coral TPU PCI Passthrough: Frigate.nvr which is one of the popular uses for the Coral do not recommend it for new installs either. Frigate updated their recommendations because of the Google decision to sunset the device and because there are alternative options available to frigate for image inferencing. The coral is still supported though, and frigate is not the only use case or platform that can benefit from an accelerator. At the end of the day if you've already got the hardware, and it's efficient enough to run, then not using it is a waste or resources that could be allocated to other VMs.

Thank you all for information. I will try to virtualize GPU to Windows VMs.

@markxc could you tell us what endpoint you used ?

To elaborate: it was already in progress, but delayed due to 1. the fact that I underestimated the impact (I thought we'd overwrite the file only when we'd modify it, not every update the package) and 2. lots of other priorities keeping the team busy. But it's clearly the reminder, and the help in better diagnosing it, that allowed to have it re-prioritized and (partly, we've mostly bought time to fix it properly, by simply avoiding to touch the file at all at the moment) fixed so fast.

I made a list of processes with a small bash script that I am attaching here [image: 1769612600385-screenshot-2026-01-28-alle-16.01.53.png]

Thank you! So in this case, if better option for me to join the hosts to existing pool and then shutdown VMs and start it on host with older CPUs. Thank you again!

For documentation purposes, there's a more general step of "Any VM you can shut down, do". Live Migration is great for VMs which need to stay up, but it's not free, and not even cheep. You will get done quicker if you can shut down VMs you don't need, migrate fewer things, and then (re)boot everything at the end.

I just learned something new, thats awesome

@dinhngtu I believe it was changing the guest TSC Mode that seems to have done the trick.