Hi, everyone. Nice to see this project turning into reality. I will try to spend time here as possible, which is hard with already being spread thinly. I've been a XenServer user for around a decade and am as interesting in learning as well as contributing whatever knowledge might be helpful to the community.

Best regards,
-=Tobias

@olivierlambert said in NUMA-impact - Xeon/Epyc - 1P vs 2P:

There is no universal answer (because it's mostly depending on your VM load and what do you expect). As usual, my advice is to keep it simple if you don't have a problem with it (ie: you are satisfied by the perf.). Even a default EPYC configuration will be likely always better than a Xeon one.

After that, if you want to go deeper and learn the details, it's OK, let me just ping @tjkreidl who did a remarkable job (if I remember correctly) on this very topic.

Thanks for the mention, @olivierlambert ! Here's a link to part 3, which contains links back to parts 1 and 2. Note that NUMA will affect EPYC processors differently as they changed the die configuration at one point with the number of cores. I'm open for any questions on this topic. https://blogs.mycugc.org/2019/04/30/a-tale-of-two-servers-part-3-the-influence-of-numa-cpus-and-sockets-cores-persocket-plus-other-vm-settings-on-apps-and-gpu-performance/

@epretorious I would add that you have to be careful about overprovisioning when NUMA/vNUMA kicks in, that is when you allocate more VCPUs to exceed the number of physical CPUs of a bank of them as well as the associated physical memory (assume, for the sake of argument, you have two banks of physical CPUs and each has directly accessible to it one of two banks of memory) then things get inefficient because a CPU may need to go across to a different bank of memory to access data and there is additional overhead involved. See for example this article and the two preceding it:
https://blogs.mycugc.org/2019/04/30/a-tale-of-two-servers-part-3-the-influence-of-numa-cpus-and-sockets-cores-persocket-plus-other-vm-settings-on-apps-and-gpu-performance/

-=Tobias

@planedrop CPU compatibility is automatic now, albeit the CPUs have to be somewhat similar. That said, Intel and AMD CPUs cannot be mixed. There used to be scripts out there that you could use to check compatibility but I have not seen them for quite some time. Note that the CPU mask will be reduced to the lowest common denominator -- that is to say, the oldest, least-featured CPU in the pool -- so I would be careful what servers you include in your pool(s).
See also: https://xcp-ng.org/forum/topic/6896/non-server-cpu-compatibility-ryzen-and-intel/21

And from the CLI:

1. xe host-list (to get the UUID of the host)
2. xe pool-eject host-uuid=<host_UUID>

@jhansen I'd suggest booting off some Linux rescue disk and just redoing all the partitions manually. I ran into an upgrade issue once on a Dell box because of some stupid little extra utility partition they had in there. See: http://xenserver.pl/author/tobias-kreidl
Look for "The (In)famous Dell Utility Partition."

Tapdisk is just palin not good as a storage control mechanism, IMO. There's only so much you can pull out of it.

@Andrew Thank you for that, much appreciated. Although I was aware of this process for SSD drives, I did not know that spinning disks had become that much smarter in the interim (~40 years!). But in any case, raw drives are very powerful if you have decent code to access them and the overhead can be appreciably less than with formatted drives.

@gustavoninetyone -- it's not on the supported hardware list, hence highly unlikely. See Olivier's response near the top of this category to see what GPUs are supported.
-=Tobias

@olivierlambert Indeed. NVIDIA in particular is bent on your having to license any sort of vGPU technology and it would not surprise me to see even passthrough leveraged at some point in the future. There are already restrictions on being able to make use of Quadro features, for example, of you don't have licenses.

Also, note that you need one video board reserved to run the native OS (XCP-ng) dom0 instance itself, so the server has to effectovely have at least two graphics boards such that one can be used as a GPU.

@kevdog You can force a host to be ejected even if the host is not reachable any longer from the pool master using:
xe host-forget uuid=UUID
If there is any issue because VMs are thought to be running still on that host,
you may need to do a power state reset on such before you can get rid of that host.
See if that works for you.

@planedrop CPU compatibility is automatic now, albeit the CPUs have to be somewhat similar. That said, Intel and AMD CPUs cannot be mixed. There used to be scripts out there that you could use to check compatibility but I have not seen them for quite some time. Note that the CPU mask will be reduced to the lowest common denominator -- that is to say, the oldest, least-featured CPU in the pool -- so I would be careful what servers you include in your pool(s).
See also: https://xcp-ng.org/forum/topic/6896/non-server-cpu-compatibility-ryzen-and-intel/21

@christopher-petzel Sometimes this can happen if the host and VMs do not use the same NTPP server(s) or are not syncing properly with them, and therefore fail to keep the times all properly synchronized. I'd check to make sure all are in sync.

@nhoty Sadly, it has been announced that the NUC will be discontinued by Intel, but ASUS is evidently licensing them and will likely keep making similar products. https://www.theverge.com/2023/7/19/23800006/intel-nuc-computers-asus-license-support

@KPS You could put in a task list in the local startup script. See various options here: https://www.baeldung.com/linux/run-script-on-startup

@aHguf5QP Xen guest tools can make quite a difference in performance. See if that helps.
Good to hear dom0 is on a totally separate device than the one that contains any VMs.

@aHguf5QP Sharing the VCPUs with the VM may be art of it, esp. if dom0 sometimes runs 150%. You may be just up against the limitations of the system.
XenTools (guest tools) is installed OK, right? Diagnosing slow performance can be a long and tedious process. You can google for various suggestions, but each system is unique.
The system disk is the same as used for the VM disk? That could be also somewhat of a limitation since they haave to share the I/O from the same physical device..

@aHguf5QP 2.4 GHz is pretty slow by current standards, but should be OK for a single VM. How many VCPUs are dedicated to dom0?
As to I/O, dd is not a very accurate testing mechanism, but may not be the reason for slowness.
With the I/O, iostat should show if there are bottlenecks.
Storage should always be on a separate subnet as a VM or any other networks.

@aHguf5QP Did you try running top and xentop from the host command line to see about resource utilization (CPU and memory), as well is iostat to look at the storage I/O resource utilization? It may take more memory and/or CPU cores for dom0.
Some of slowness can be from lack of resources, some can be inherent in the storage (RAID configuration, connectivity bottlenecks, buffer and other settings, etc.). If this is a single SSD drive then you're at the mercy of what the controller and drive itself can manage on the storage end, but there may be system settings that could help.
Just 4 cores and 8 threads is not much for running a server, plus you didn't mention what the clock speed is, which also factors in quite a bit in performance.

@graham_p Sounds like a configuration to do this would include the following:

1. List of hosts in order of preference the VM should run on, with automatic fallback in order of priority.
2. List of hosts to not start the VM on, no matter what
3. Means of notifying sysadmins of changes (email?) or at least add additional logging information.
   @olivierlambert , that would be great if those features could be incorporated!