@planedrop Thanks for the information! Back to the more general idea of the wide VMs, I think it was originally more of an efficiency issue. Our Support team noticed high CPU usage, but the pCPU and overall host usage was very low.

Turns out we had stacked multiple, heavy-utilized, wide VMs on the same hosts. After looking at the stats, there so was so much co-stop that they were wasting so much time on trying to co-schedule the vCPUs. After spreading out the wide VMs we actually saw the hosts overall consume more CPU and the performance issues went away.

With us getting fresh start on a new hypervisor, instilling a desire for right-sizing VMs and scaling out versus up will probably be the way to go.

Thanks again for all your help!

@planedrop Thanks for the information! Back to the more general idea of the wide VMs, I think it was originally more of an efficiency issue. Our Support team noticed high CPU usage, but the pCPU and overall host usage was very low.

Turns out we had stacked multiple, heavy-utilized, wide VMs on the same hosts. After looking at the stats, there so was so much co-stop that they were wasting so much time on trying to co-schedule the vCPUs. After spreading out the wide VMs we actually saw the hosts overall consume more CPU and the performance issues went away.

With us getting fresh start on a new hypervisor, instilling a desire for right-sizing VMs and scaling out versus up will probably be the way to go.

Thanks again for all your help!

@olivierlambert Thanks for the information! Will any newer version of the Xen Guest Tools installed continue to register as 8.3.60-1 moving forward?

Is the Xen Guest Agent ready for GA or should we hold off on installing it for now?

While I don't think this is a big deal, I thought I would bring it up.

We just installed the latest Xen Guest Tools for Linux from the Github page below. The version on Github, and when installing it via dnf on our guest, shows 8.4.0-1. When I look at the VM in the XOA web UI, it says, "Management agent 8.3.60-1 detected." I tried restarting the xe-linux-distribution service and rebooting the VM, but it shows the same version detected.

https://github.com/xenserver/xe-guest-utilities/releases

Is there just a discrepancy in the detection or is this working as intended? I don't think there is likely to be any impact but wanted to check before installing it on more VMs.

Any advice, recommendations or information is appreciated. Thanks in advance!

@planedrop One workload is a .NET application that regularly uses 80-100% of 26 vCPUs, but unfortunately, I don't know much about the application as I am not on the development team. The other is for ElasticSearch. The latter is not much of an issue anymore (I convinced them to reduce the number of vCPUs as we showed it rarely uses that much).

@Andrew Per your example, if the NUMA size is 16 cores with 32 threads, would a 32 vCPU VM fit inside the NUMA node or would that spread them out across both nodes?

@planedrop This is for wide VMs spanning across socket NUMA nodes. These are all dual-socket servers.

I was hoping if there are any best practices for topology or anything like that. I don't have any desire/intention to pin the vCPUs onto a specific NUMA node. Coming from vSphere, I was instructed, if at all possible, not to have more vCPUs on a VM than there are pCPUs on a NUMA node. That way you can let the scheduler handle it optimally.

Good Morning Everyone,

We are currently in the midst of testing XCP-ng as a replacement for our vSphere infrastructure. I have a general idea of the how wide VMs work in VMware, and what settings should be in place, for wide VMs that are larger than a NUMA node on our servers.

Are there any recommendations for VMs on XCP-ng that are span across NUMA nodes? Any advice or information would be greatly appreciated.

Thanks!