Originally an astronomer for 15 years and later, an NAU employee in IT for 25+ years, most of which as a Team Lead. I was a Citrix CTP and NVIDIA NGCA for four years prior to retirement. Over 10 years' experience with XenServer/Citrix Hypervisor and close to that with NVIDIA GRID products. I was also a Red Hat Linux administrator and system programmer. Still trying to contribute what knowledge I have for the benefit of the IT community.
Hi, everyone. Nice to see this project turning into reality. I will try to spend time here as possible, which is hard with already being spread thinly. I've been a XenServer user for around a decade and am as interesting in learning as well as contributing whatever knowledge might be helpful to the community.
Best regards,
-=Tobias
And from the CLI:
@robyt It depends on (1) licensing, if any, as some licenses go by cores vs. sockets, and (2) NUMA/VNUMA depending on how critical the performance is depending on how the VCPUs get allocated between sockets or on a single socket. Best way IMO is to try all and test with benchmarks. See, for example, this article and the previous two articles, as well as articles by Frank Denneman and others: https://blogs.mycugc.org/2019/04/30/a-tale-of-two-servers-part-3-the-influence-of-numa-cpus-and-sockets-cores-persocket-plus-other-vm-settings-on-apps-and-gpu-performance/
@olivierlambert said in NUMA-impact - Xeon/Epyc - 1P vs 2P:
There is no universal answer (because it's mostly depending on your VM load and what do you expect). As usual, my advice is to keep it simple if you don't have a problem with it (ie: you are satisfied by the perf.). Even a default EPYC configuration will be likely always better than a Xeon one.
After that, if you want to go deeper and learn the details, it's OK, let me just ping @tjkreidl who did a remarkable job (if I remember correctly) on this very topic.
Thanks for the mention, @olivierlambert ! Here's a link to part 3, which contains links back to parts 1 and 2. Note that NUMA will affect EPYC processors differently as they changed the die configuration at one point with the number of cores. I'm open for any questions on this topic. 
https://blogs.mycugc.org/2019/04/30/a-tale-of-two-servers-part-3-the-influence-of-numa-cpus-and-sockets-cores-persocket-plus-other-vm-settings-on-apps-and-gpu-performance/
@epretorious I would add that you have to be careful about overprovisioning when NUMA/vNUMA kicks in, that is when you allocate more VCPUs to exceed the number of physical CPUs of a bank of them as well as the associated physical memory (assume, for the sake of argument, you have two banks of physical CPUs and each has directly accessible to it one of two banks of memory) then things get inefficient because a CPU may need to go across to a different bank of memory to access data and there is additional overhead involved. See for example this article and the two preceding it:
https://blogs.mycugc.org/2019/04/30/a-tale-of-two-servers-part-3-the-influence-of-numa-cpus-and-sockets-cores-persocket-plus-other-vm-settings-on-apps-and-gpu-performance/
-=Tobias
@utmoab Strange, but I've seen some tasks "stuck" that could not be cancelled for some reason or aanother. A reboot is a drastic measure, but unfortunate;y sometimes the only recourse.
@kevdog You can force a host to be ejected even if the host is not reachable any longer from the pool master using:
xe host-forget uuid=UUID
If there is any issue because VMs are thought to be running still on that host,
you may need to do a power state reset on such before you can get rid of that host.
See if that works for you.
@planedrop CPU compatibility is automatic now, albeit the CPUs have to be somewhat similar. That said, Intel and AMD CPUs cannot be mixed. There used to be scripts out there that you could use to check compatibility but I have not seen them for quite some time. Note that the CPU mask will be reduced to the lowest common denominator -- that is to say, the oldest, least-featured CPU in the pool -- so I would be careful what servers you include in your pool(s).
See also: https://xcp-ng.org/forum/topic/6896/non-server-cpu-compatibility-ryzen-and-intel/21
@olivierlambert @joehays Easiest backup to a remote location IMO is to NFS-mount a drive from some other system and back up to it. That way, it can always be exported to anywhere needed, even if your server(s) are destroyed and you have to start over from scratch.
@jhansen I'd suggest booting off some Linux rescue disk and just redoing all the partitions manually. I ran into an upgrade issue once on a Dell box because of some stupid little extra utility partition they had in there. See: http://xenserver.pl/author/tobias-kreidl
Look for "The (In)famous Dell Utility Partition."
@bpsingh76 You should be able to run "xe vdi-list" on these and see if they are attached to anything. If not, it would appear they are just leftover templates.
@stevewest15 Looks to me like all your hosts might not have the same updates/patches applied. I'd check to make sure they are all up-to-date and the same on all your hosts.
@Forza That all is good advice. Again, the showmount command is a good utility that cam show you right away if you can see/mount the storage device from your host.
@CodeMercenary Sorry first off to hear about your personal issues. They need to take precedence.
If there is network contention, it's important that things not be impacted on backups because they utilize a lot of resources: network, memory, and CPU load.
That's why we always ran them over isolated network connections and at times of the day when in general VM ctivity was at a minimum. Make sure you have adequate CPU and memory on your hosts (run top or xtop) and also, iostat (I suggest adding the -x flag) can be very helpful is seeing if other resources are getting maxed out.
@CodeMercenary THis is unfortunate news. We ran backups over NFS successfully for years, but with Dell PowerVault units NFS-mounted on Dell Linux boxes and with XenServer hosted on Dell PowerEdge servers, so everything was Dell which probably made things more compatible.
You don't have some weird firewall or other authentication issue? And is your backup network on a separate physical network or at least a private VLAN?
I will also note that some problems we had now and then were due to bad Ethernet cables! Are your network connections to the storage devices bonded or using multipath?
@CodeMercenary Ouch. Make sure all your servers are properly time synchronized. Can you do a showmount from your server to the NFS storage device to see if the host has access permissions?
@CodeMercenary How odd, unless the default NFS version somehow changed since it last worked and you had to specify V3 to get it to work again.
I'd contact Synology to find out if perhaps the storage unit needs a firmware upgrade to support V4? Perhaps they've had similar feedback from
other customers.
@CodeMercenary Just a guess, but is there any NFS lock perhaps involved because of the CPU getting pegged and there being a timeout of sorts? Check the lockd daemon state, perhaps.
@msupport Many thanks for your write-up! Have you experienced any issues communicating with the NVIDIA license server?