@jbamford
Hello, can you give the output of xl info in Dom0 ? If you see hvm_directio in front of virt_caps, then PCI Passthrough is supported (i.e you have VT-d).

You can also check xl dmesg for vt-d specific messages (you can also give the output of it).

@EddieA Can you try differents combinations of passedthrough hardware in this VM ?

e.g try with each device one by one at a time; at least in the VM

@benapetr
Consider making a PR to https://github.com/xcp-ng/xcp-ng-org (https://github.com/xcp-ng/xcp-ng-org/blob/master/docs/management/management.md) if you want to make your project more visible.

@nvs can you try with a serial console and something listening on it; so when it crashes, we get the crash reason ?

@dcskinner @olivierlambert

You can read key/values from the xenstore, and write some (from VM to outside), but you cannot write values "in live" from outside the VM to the inside.

It is, but XAPI doesn't provide a interface for it.

do the guest tools quiesce the filesystems before snapshotting?

Tools are aware of a snapshot so you don't have blocks in flight.

do the guest tools quiesce the filesystems before snapshotting?

Guests kernel are aware, as it is them that are performing a "suspend" on toolstack request (thus quiece filesystems); although "tools" can only observe that the system has been suspended after the fact by measuring side effects, and not orchestrate it.

It's because suspend/resume operation doesn't come from "guest tools" actually, but instead from the kernel drivers. So userland tools has no say on it.

@antest can you try running the VM with only 1 vCPU ?

@antest can you retry after doing /opt/xensource/libexec/xen-cmdline --set-xen iommu=debug then rebooting.
And also reporting full DomU and Dom0 dmesg (in addition to xl dmesg) ?

Do you get anything special written to xl dmesg in Dom0 ?

@hitechhillbilly no it doesn't, it just ensures the N-th vCPU of Dom0 only runs on N-th pCPU of the machine.
Not sure about the practical impact of it, in the past it has been used for getting meaningful CPU temperatures from coretemp (with physical core matching virtual one), but that doesn't work anymore since Xen filters MSR accesses (including Dom0).

@dlucas said in XCP-NG 8.3 Crash on Shutdown - HP DL360 g9:

Have checked the IML log and the only thing that is being logged is this:

"13","Critical","PCI Bus","12/04/2025 00:03","12/04/2025 00:03","1","PCI Bus Error (Slot 0, Bus 0, Device 2, Function 2)",
"12","Critical","PCI Bus","12/03/2025 23:01","12/03/2025 23:01","1","PCI Bus Error (Slot 0, Bus 0, Device 2, Function 2)",
"11","Critical","PCI Bus","12/03/2025 22:53","12/03/2025 22:53","1","PCI Bus Error (Slot 0, Bus 0, Device 2, Function 2)",
"10","Critical","PCI Bus","12/01/2025 23:44","12/01/2025 23:44","1","PCI Bus Error (Slot 0, Bus 0, Device 2, Function 2)",
"9","Critical","PCI Bus","12/01/2025 01:31","12/01/2025 01:31","1","PCI Bus Error (Slot 0, Bus 0, Device 2, Function 2)",

I thought that a shutdown command from the command prompt solved the issue but it seems to always reboot when ever you tell it to shutdown.

What is device 00:02.2 (through lspci) ?

@SethNY said in Windows 11 WSL2 is not supported with your current machine configuration:

XCP-ng 8.3, XO from sources.
Created Win11 from ISO using the built-in Windows 11 template
Configured and turned into template for cloning.

Trouble installing WSL on the cloned Win11:
WSL2 is not supported with your current machine configuration

This worked a couple years ago Win10 on XCP-ng 8.2, Ubuntu 22.04.

From administrative powershell

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
(Accept the reboot, back to administrative powersehll)
wsl --install
(fails)
wsl --list --online
wsl --install -d Ubuntu-24.04
(fails again)

I tried enabling Nested Virtualization for the VM without success
Booted to (F2) BIOS and confirmed not virtualization options there to enable

Has anyone got Win11/WSL/8.3 working? I'm hoping it's not due to not installing a Windows license key.

You were probably using WSL 1 before, and now using WSL 2 (which requires Hyper-V thus nested virtualization). And I'm not aware of WSL 2 working on XCP-ng 8.2, I assume you were actually using WSL 1 previously.

You can use WSL 1 by using

wsl.exe --set-default-version 1

and

wsl.exe --set-version <Distro> 1

@McHenry said in pfSense Guest Tools:

I have been using pfSense with xcp-ng for a while now without installing the guest tools.

Due to some networking complications I have decided to install the guest tools to eliminate this as the cause.

Q1) Are the guest tools required on pfSense and what do they do?

Actually no, their main functionnality is to provide some data (memory usage, IPs) to XCP-ng to report them upward, AFAICT, it doesn't impact behavior in a significant way. It's actually not "PV drivers".

Q2) Are these tools being maintained?

This version of the guest agent is very limited and ancient (I don't exactly what it does, but not much IIRC).

We're working on a new guest agent in Rust, which actually works pretty well on Linux, Windows (actually used in latest Windows drivers) and FreeBSD (NetBSD is also possible, but current code lacks some platform-specific bits). But we still need to sort-out some issues (some technical and some others non-technical) before making it broadly available / making it packaged.

@Maelstrom96 said in Epyc VM to VM networking slow:

What is the exact kernel patch that is required for the xen-platform-pci-bar-uc=false fix to work on a Linux guest? We're looking at potentially compiling our own kernel with the xen-netfront.c patch, and we would like to see about adding the other part of the Kernel code needed for the Grant table fix.

Patch is in Linux since 5.19-rc. You also find it in some stable branches like 5.15.

Otherwise, you can check this patch https://lore.kernel.org/xen-devel/ea4945df138527ed63e711cb77e3b333f7b3a4c9.1751633056.git.teddy.astie@vates.tech/

@joeymorin said in Building from source fails with commit cb96de6:

Greetings all,

Subject pretty much says it. Build output goes sideways starting with:

@xen-orchestra/qcow2:build: src/disk/QcowDisk.mts(94,52): error TS2365: Operator '&' cannot be applied to types 'bigint' and 'number'.

Github CI is failing on this commit, try reverting to a commit that pass CI.

@rk9268vc said in Having issues installing StartOS as a VM. Cant detect a disk for it to install to.:

@TeddyAstie so can i just not run this OS on xcp-ng? is there no workaround? Would this run on proxmox?

you can try adding to Linux command-line (in grub) xen_emul_unplug=never to disable PV drivers, thus making udev see ATA/NVMe drivers, maybe that's enough as long it's only the installer

@rk9268vc said in Having issues installing StartOS as a VM. Cant detect a disk for it to install to.:

@TeddyAstie so is this fixed in a newer version of xcp-ng? and if so, can i update live?

it's not fixable from XCP-ng side, it is a bug in Debian/others...

This happens because StartOS uses /dev/disk/by-path to get disk list and due to missing rules in 60-persistent-storage.rules (thus systemd/udev), Xen PV disks don't get their disks populated in /dev/disk/by-id.

In other words, it is a systemd bug which wants to be fixed, but getting that backported to older distros is gonna get tricky.

@Maelstrom96 said in Kioxia CM7 PCIe pass-through crash:

@TeddyAstie It's a newly deployed Rocky Linux 9.6 with all the latest updates applied to it.

Nested virtualization is disabled.

Can you do

xl debug-key G
xl debug-key +

and try restarting the guest to try having more informations.

Also, note that your installation is not up to date (even though I don't think it is going to change anything).

What guest you are trying to boot ?
Is nested virtualization enabled for this guest, if it is enabled, does that work if you disable it ?

@mickwilli said in Cannot Install Windows 10 in New VM:

@dinhngtu Thanks. I'll try and give that a go in the next few days and will report back.

Note that I have Windows 10 1803 installed without any issues. Trying to update to 22H2 fails with the same blue screen error as you'd expect. Sadly, no crash dump as it seems to fail very early in the boot process.

Do you get any additional BSOD details in xl dmesg ?