@Kajetan321 Your disk image is presented as a block device to the VM. This is done by a kernel module called blktap and its userspace part tapdisk.

You are seeing dom0's kernel messages when these block devices are attached by blktap. It's just an informational message.

@plaidypus I am not 100% sure what the correct answer is for the default XCP configuration. I think the basic answer is: no. Xen/XCP does not care what cores it uses for your VM. So on average your performance will be a little worse than not crossing NUMA nodes but better than always interleaving NUMA nodes. Some systems will be better/worse than others.

Xen/XCP hypervisor does have a NUMA aware scheduler. There are two basic modes, one is CPU hard pinning where you specify which cores a VM (domain) uses. This would force the VM to use only the cores it is assigned. The other is to let Xen/XCP do its own work where it tries to schedule core use of a VM (domain) on a single CPU pool. The problem with this is the default config is to put all cores (and HT) in a single default pool. There are some options to try and enable best-effort NUMA assignment but I believe it is not set that way by default.

You can configure CPUs of a NUMA node into an individual pool (see below). A VM can be set for an affinity for a single pool (soft CPU pinning). This would keep most of the work on that single node as you want.

The links listed before to have good information about NUMA and CPU pinning. Below are some more:

Here is an older link about Xen on NUMA machines.
Here is a link about Xen CPU pools.
Here is a link about performance improvements on an AMD EPYC CPU (mostly related to AMD cache design).

There are also APIs in the guest tools to allow the VM to request resources based on NUMA nodes.

If you start hard limiting VMs where/how they can run you may break migration and HA for your XCP pool.

@R2rho yeah, there are Supermicro systems with AM5 which can handle a decent amount of load, like based on the h13sae-mf, like:

https://www.supermicro.com/de/products/system/mainstream/1u/as-1015a-mt
(with less depth)

Seem to be stable, but we have a small issue regarding onboard graphics ATM:

https://xcp-ng.org/forum/topic/9976/black-screen-after-install-on-supermicro-h13sae-mf-with-ryzen-9950x/3?_=1734419502978

You should start by checking the logs. https://docs.xcp-ng.org/troubleshooting/log-files/

Thanks for letting us know, and I'm happy you have thing working nicely now.
I think to mark this as resolved you need to convert your original post as a question, and it can then be marked as resolved. I actually cannot do it myself, I think only a few people have the permission to do it for others at Vates.

Can you try on latest release channel and see if you still have the error? Thanks!

Enjoy XO & XCP-ng then 🙂

@olivierlambert

I resolved it, logged in the XCP-ng console and set the NTP servers to default. All good now.
Thanks

xcp-ntp2.png

I have yet to successfully reboot a host from XO. I have tools installed on three out of four VM. I try shutting down the VM without tools, then issue reboot host in XO GUI. Get warning:
Some VMs cannot be migrated without first rebooting this host. Please try force reboot.

Is this a setting perhaps?

I'm always needing to shutdown all VMs before I can reboot the host. It is a single host environment, so there will never be an available server to migrate VMs to.

Rebooting in XSCONSOLE causes the host to be unavailable in XO. I have to remove then add the host back in XO.

Can anyone shed light as to what I'm doing wrong?

Do I just need to always run the following after a reboot?
xe host-enable host=xxxx

Preferably after, since the PV drivers contain some VM-specific settings that may be wiped by Sysprep.

FWIW...having the same issue with Server 2025.

@dinhngtu said in Issues with XCP-ng Windows PV Tools 9.0.9000.0 Testsign Pre-release:

With 9.0.9000.0, you could sometimes run into this issue if you reboot too early after installing. This will also be fixed in the next release.

I just did two more tests and the boot-up failure happened both times. On the last test I waited some time (like 30 to 60 sec) after the driver installation before rebooting (using Windows restart from the start menu). Even though I waited before rebooting I still encountered the boot-up problem:

Windows is spinning on this screen and doesn't finish booting:
ef30a370-5897-41e6-a0f1-41fa3720dacb-image.png

(In regards to uninstalling XenServer / Citrix Tools)

I tested uninstalling XenServer tools v9.4.0 on Windows and it seems to uninstall cleanly. (the xcp-ng docs mention there have been problems uninstalling everything cleanly in the past)

Also since XenServer tools v9.3.1 there is a new purge function:

uninstall.exe purge - (9.3.1 and later) Resets the VM to a clean state as it was before any part of the XenServer VM Tools were installed.

Full uninstall of all XenServer VM Tools components
https://docs.xenserver.com/en-us/citrix-hypervisor/vms/windows/vm-tools.html#full-uninstall-of-all-xenserver-vm-tools-components

What does the purge option remove?
https://docs.xenserver.com/en-us/citrix-hypervisor/vms/windows/vm-tools.html#what-does-the-purge-option-remove

I wasn't aware of this plugin even existing (thanks for the discovery). I'm not sure it's really maintained, but we have a clear goal of getting one officially maintained anyway.

@olivierlambert

Which one, powershell or WAC. They are kind of one in the same when you look at how WAC makes the calls to do the work, there's a button to reveal the powershell behind the button clicks so you might be able to get both at once.

But WAC draws nice graphs and gives us a GUI to get the work done. Yes I know, real Admins don't use GUI. Whatever, AI will eat our jobs anyway, just trying to stick it out another 9 years at this point.

When I get time, I'll still probably take a swing at this task for WAC, because to me this would be valuable. Big learning curve though.

Now if MS would finally add Group Policy to WAC and fix some of the DHCP and DNS features that are still in preview...

@dinhngtu Ah thank you! Idk why I couldn't find that on my own, I searched for a while.

According to the documentation, you only need xfsprogs.

I'm not aware of such problem it's like the VM was kept in limbo. You did the right thing with your commands.

Hello,

I've tried benchmarking several Windows VMs from another client and am getting expected performance:

Read: .\diskspd.exe -c1G -d10 -Su -Sw g:\data.dat Write: .\diskspd.exe -c1G -d10 -Su -Sw -w100 g:\data.dat Windows Server 2022: Read IO thread | bytes | I/Os | MiB/s | I/O per s | file ------------------------------------------------------------------------------ 0 | 1151533056 | 17571 | 109.72 | 1755.53 | g:\data.dat (1GiB) ------------------------------------------------------------------------------ total: 1151533056 | 17571 | 109.72 | 1755.53 Write IO thread | bytes | I/Os | MiB/s | I/O per s | file ------------------------------------------------------------------------------ 0 | 966000640 | 14740 | 92.12 | 1473.98 | g:\data.dat (1GiB) ------------------------------------------------------------------------------ total: 966000640 | 14740 | 92.12 | 1473.98 Windows 11 24H2 (with Eset): Read IO thread | bytes | I/Os | MiB/s | I/O per s | file ------------------------------------------------------------------------------ 0 | 1100283904 | 16789 | 104.81 | 1677.01 | g:\data.dat (1GiB) ------------------------------------------------------------------------------ total: 1100283904 | 16789 | 104.81 | 1677.01 Write IO thread | bytes | I/Os | MiB/s | I/O per s | file ------------------------------------------------------------------------------ 0 | 931594240 | 14215 | 88.82 | 1421.07 | g:\data.dat (1GiB) ------------------------------------------------------------------------------ total: 931594240 | 14215 | 88.82 | 1421.07

Several questions:

Are you running benchmarks on a separate virtual disk? If not, could you try again using a share created from a separate virtual disk? Could you run CrystalDiskMark locally on these VMs and see the performance? You mentioned that Eset security takes up 100% CPU during the benchmark. Could you try temporarily uninstalling Eset and testing again?