Thanks for the clarification, @dsmteam !

Hi Aleksander,
I noticed you were on tid 11869 back in February, where @fatek shared his working setup on a Tesla P4 with the NVIDIA-GRID-XenServer-8 driver, and @tjkreidl ran through the cards that have been reported to work (Tesla M6/M10/M60, P4/P6/P40/P100, V100, T4, A2/A10/A16/A40, RTX A5000/A6000/6000/8000). You closed that thread with "I will try to virtualize GPU to Windows VMs." I'm curious what happened: did you give it a try and hit something specific, or is this a fresh use case where the constraints are different?

The reason I'm asking rather than starting from scratch: fatek was pretty clear that the path is YMMV (XenServer is the supported hypervisor for those drivers, XCP-ng isn't officially), and the XCP-ng vGPU docs confirm it. So if you tried and hit a wall, that's the conversation worth having here. If you tried and it worked but you're now scaling to more Windows VMs and want to know if the same setup holds at 4+, that's a different conversation. On the supported-on-paper side, the same docs page says AMD MxGPU is "trivial using industry standard" if AMD hardware is on the table.

I don't have hands-on with any of this myself, but it'd help to know which problem you're trying to solve now versus February.

Thanks!

If the lower retention value gets things stable, that probably confirms Pilow's hypothesis. If it doesn't help, that's the signal that something heavier is going on, and a @Team-XO-Backend ping would make sense. Would you mind dropping the result back here either way? Helps the next person hitting the same wall.

The sr.scan-driven SMlog growth angle that gumbo2k surfaced is a real lead; there's some context in the storage-related log files reference, but the docs don't go as far as "here's how to throttle it safely on a pool where the underlying disks should spin down."

Soft ping to @Team-Storage and @Team-Hypervisor-Kernel: could one of you weigh in on whether other-config:auto-scan=false on the SR is the supported way to reduce scan pressure, or if there's a better lever? I don't want to send anyone down a path that breaks an SR. Apologies if this has already been answered somewhere I haven't seen.

That GitHub HTML-bundle pain is real, especially when whatever exported the pages dragged in half the surrounding site CSS and sidebar widgets along with the article. You spent the time writing the originals; the file-wrangling shouldn't be what costs you another evening.

If it's useful, I'm up for taking the cleanup off your hands. Concretely:

• pull the HTML you saved from the repo (or a zip / branch you point me at)
• strip out the noise: scripts, sidebars, navigation, anything that isn't the article body and its images
• convert to markdown, one file per article, images kept alongside
• open a PR against tobiaskreidl/Citrix-Tobias-Kreidl-Collection (https://github.com/tobiaskreidl/Citrix-Tobias-Kreidl-Collection) so you stay the owner and only have to review and merge

If the saved bundles are too far gone, I'd fall back to the Citrix URLs @john.c surfaced and pull clean copies via the Wayback Machine. Same end result on your repo.

Separately, and only if that lands cleanly, I'd love to talk to Thomas Moraine and the docs team at Vates about whether parts of this could find a home on docs.xcp-ng.org, linked back to your repo with full credit. The NUMA-affinity page I pointed at earlier is shallow next to what you wrote, and the BIOS / GPU-scheduler material has no current equivalent at all. Whether the 2019 specifics still map cleanly onto current XCP-ng versions and hardware is a separate conversation; even as an archived reference, it's more than what's there today.

No pressure, no obligation. If you'd rather keep iterating on the repo yourself, that's fine too. But if a clean-the-HTML-and-PR-it pass would save you a week of GitHub-tool frustration, I'm genuinely up for it. Just point me at the bundle.

Take it with a grain of salt, but I think bonds are usually managed as a whole rather than edited port by port in the UI. As far as I can tell, the supported route is from the network section in Xen Orchestra (the bonding part of the infrastructure docs is here: https://docs.xen-orchestra.com/xo5/manage_infrastructure#network-bonding), and on the CLI side, the bond commands are documented at https://docs.xcp-ng.org/appendix/cli_reference#bond-create (there's a matching bond-destroy command alongside it).
My honest guess is you may end up destroying and recreating the bond with the four ports you want to keep, since I'm not sure removing a single member in place is exposed anywhere, but I could easily be wrong.
If there's a cleaner way that avoids the recreate, someone will let us know.

Thanks for taking the time to write this up with before/after numbers. That kind of post is genuinely useful for the next person hitting slow backups.
Pilow's catch looks like the important one to me: I think raising the VM's RAM on its own doesn't help much unless xo-server is also told it can use it, which is the heap-limit tip here: https://docs.xen-orchestra.com/xo5/troubleshooting#memory.
I could be wrong, but the recommended XOA sizing also lives at https://docs.xen-orchestra.com/xo5/xoa#xoa-vm-specifications if you want to sanity-check vCPU/RAM against what's suggested for your backup load.

Good to hear acebmxer confirmed a fresh 6 GB deploy sizes the heap correctly by itself; that matches my (limited) understanding that the manual step mostly matters when you grow the VM later.
Let's wait for the experts to chime in; they will know far more about squeezing the last bit of backup throughput out of XOA.

This is great to see, thank you for taking the time to rescue this; and thanks to @john.c for the recovery work and to @tjkreidl for writing it in the first place.
I went looking, and there is a small XCP-ng-specific piece on this in the official docs under NUMA affinity (https://docs.xcp-ng.org/compute#numa-affinity), but it's nothing like the depth of the Tale of Two Servers series, so having the originals archived is genuinely useful.
I won't pretend to judge how much of the 2019 BIOS and GPU-scheduler guidance still maps cleanly onto current hardware and XCP-ng versions; others here will know where it's aged and where it hasn't.
I'll make sure this is on our radar on the docs side, because it keeps coming up. Really appreciate you keeping this from disappearing.

This is a nice bit of community tooling, and thanks for keeping it updated and being upfront about the "use at your own risk" part.
On the naming point a few people raised, I think the distinction folks are drawing is fair: XOA is the Vates-built appliance, and anything you compile yourself is XO from sources.
The official docs cover that path at https://docs.xen-orchestra.com/xo5/installation#from-the-sources, including the note that there is no pro support for that method.
Hope the project keeps going well!

Thanks for the ping and for narrowing it down; that live-migration repro is a really useful signal.
I don't know enough about how the guest tools report IPs back through XAPI to say where the canonicalisation should happen, but it sounds like something @Team-Hypervisor-Kernel might want to look at since the trigger is on the agent side after migration.
If it turns out to be reproducible on another Windows guest version (2022, 2019), that might help narrow it further; no pressure though, you've already done the hard part.

Thanks a lot for the feedback!

Hi!
I think what you're seeing may be stale entries in the XAPI database, ghost records that can survive upgrades or host reconfigurations.
From what I've read, xe vm-destroy uuid=<vm-uuid> removes the record without touching any storage, which seems like what you need here; the xe CLI reference confirms storage is left intact.
I think you can get the UUID first with xe vm-list name-label="Before Ubuntu Update" (replacing the name with whichever one you're after).
I'm not entirely sure why xsconsole would show them but XO wouldn't, so if the VMs don't turn up in xe vm-list, it might be worth a mention to Team-XAPI-Network, they'll know the right way to dig into XAPI state.

Hey, thanks for flagging this. I think what may be happening is that after the merge at the end of the retention chain, the UI might be picking up the full VM size (2.17 TB) instead of the actual KEY file size on disk (9.1 GB).
If you can confirm the key file is genuinely 9.1 GB on your storage, that would point to a display bug rather than a data issue.
Could you please check, and if so, it might be worth opening an issue on the XO GitHub with the numbers? I'm not entirely sure of the exact mechanism here, so @Team-XO-Backend would know better than me.

As far as I understand, @john.c has it right: both the pool-level and VM-level flags need to be on for it to work.

On what actually triggers it (or what I think I understood): it's XAPI itself running on the host, not XO. XO is just showing you a toggle for a value that lives in the hypervisor config, so if XOA goes down it doesn't affect the auto-start behaviour, right?

For start order: I honestly don't know enough about this part to give a confident answer, but I think there's a start_order parameter in XAPI's other-config for each VM that controls it.
Someone more familiar with the internals here would know for sure. The main docs don't seem to cover it well, which might be worth flagging.

https://docs.xen-orchestra.com/xo5/manage_infrastructure#auto-power-vm
https://docs.xcp-ng.org/guides/autostart-vm#with-the-cli

The "PIF is not attached" usually means the network interface selected as the migration network isn't currently active on the target host.
It can happen after upgrades if the host hasn't been fully rebooted.
Worth checking whether a reboot of both hosts changes anything, and running xe pif-list to see whether that specific PIF shows currently-attached: true on the target.
If the PIF looks attached in xe but migration still fails, might be worth a ping to Team-XAPI-Network.

From what I understand, the OS-to-iDRAC pass-through is a direct USB link between the host and the BMC, designed for the hypervisor host itself to talk to the iDRAC, not something that would naturally bridge through Xen's networking layer into a VM.
That might be why the PIF shows up in XCP-ng but the VM still can't reach 169.254.0.1; the bare metal case works because the OS sits directly on the hardware with no virtualisation layer in between. If you need the VM to have direct BMC access, USB passthrough might be worth exploring; XCP-ng does
support passing USB devices through to VMs (https://docs.xcp-ng.org/compute#usb-passthrough), though I'm not certain the iDRAC virtual USB NIC would show up as a passthrough-able device.
Might be worth a mention to @Team-Hypervisor-Kernel to find out; others here will know more than me.

From what I understand, when XCP-ng reverts to a snapshot it restores the full VM state from that point (metadata included, not just the disk contents) so the creation date field would get rolled back along with everything else; that might be why it now matches the snapshot timestamp rather than
the original. I might be wrong about the internals though.

It's a bit confusing if you were relying on that field to track VM history, and I don't think https://docs.xen-orchestra.com/xo5/manage_infrastructure#snapshot-management covers this explicitly.
Might be worth a mention to @Team-Documentation-Knowledge-Management; it's the kind of thing that catches people off guard because nothing warns you upfront that metadata rolls back too.
My $0.02.

@MajorP93: Thanks for expressing yourself regarding that, and I'll be transparent about the thinking behind it.

Filing those on GitHub isn't me asking anyone to stop using the forum, quite the opposite in fact.
The forum is where real conversations happen, and that's valuable in a way a GitHub issue never quite is. But forum threads scroll, get buried, and developers can't easily maintain a stable backlog out of them. GitHub gives the team a place where things don't disappear or get buried.

Think of it as belt and suspenders (which I need now that I'm getting old ). The discussion lives here, the tracking lives there. My goal as community manager is to be the relay between the two, so you don't have to worry about it.
File things here, talk about them here, and I'll make sure what matters makes it into the right repo.
Or at least, that's the plan. Mine.

@escape222 said:

Slow booting on Debian 13 VM created from a template

I recently tried cloning a VM from a template (created from a full install of Debian 13). What was noticed was that the system takes forever to boot when the VM is created from XO-6. The issue does not happen in XO-5. The VM seems to hang at the TianoCore boot screen .

What I noticed is that when the VM is cloned with XO-6 the boot order somehow changes to Network boot as the first option

This does not happen (change of boot order) if the VM is cloned from XO-5 and there is no boot delay. The boot process seems to wait for more than 2 minutes before it fails network boot and then proceeds normally to boot from the Hard drive

Thanks

This one's filed as GitHub issue #9802. The clone inserting Network Boot as the first option is reproducible and @MajorP93 confirmed the extended boot time with spinlock messages too, which strengthens the report. Both symptoms are in the issue. If you find a workaround in the meantime (like manually reordering boot entries post-clone), adding it to the issue would be useful for others hitting the same thing.

@Greg_E said:

@MajorP93

Before the LTS release UEFI worked extremely well, something kind of went sideways when it hit release. I only have a single UEFI VM, it was created under version 8.2 and does not give me trouble.

I guess I need to get my lab back up, had a UPS fail and shut everything off a few days ago. Then I need to test UEFI VMs and see what I can see. But back when it went LTS I reinstalled from scratch and tried to do everything through XO-lite to get the first VM running and found that Debian 13 or Windows Server 2022 would not boot if I created them with UEFI. Since then I haven't tested this again.

I'm also tempted to just move my whole lab to version 9, waiting for the next ISO to land and I may do this. I can run real testing workloads on Harvester for the time being and maybe play around migrating back and forth between both systems.

Logged this as GitHub issue #9801. The detail that 8.2-era UEFI VMs still work but new ones created through XO Lite don't is a useful clue for the team to narrow the scope. Worth checking if the same behavior shows up when creating the VM through XO 5 vs XO Lite, in case it's a frontend difference rather than hypervisor-level. Either way, the issue is now tracked.