@piotrlotr1 Most VMDK's should work, but the V2V tool is really what is meant for this. It lets you warm migrate from VMware to XCP-ng with very little downtime.

@piotrlotr1 Maybe I missed some context in this thread, so apologies if I did.

But the V2V tool should handle this, is there a reason you are wanting to do it as a VMDK import instead?

I've used the V2V a lot and it works quite well.

@MAnon This is a valid point actually, and without additional work, you couldn't just restore to another hypervisor.

However, check this blog post: https://xen-orchestra.com/blog/xen-orchestra-5-100/?utm_campaign=mail_5.100&utm_term=logo&ct=YTo1OntzOjY6InNvdXJjZSI7YToyOntpOjA7czo1OiJlbWFpbCI7aToxO2k6NjU7fXM6NToiZW1haWwiO2k6NjU7czo0OiJzdGF0IjtzOjIyOiI2NzIzODI1NDE4ZjVmMjE5NDI2OTYwIjtzOjQ6ImxlYWQiO3M6NToiODM5ODciO3M6NzoiY2hhbm5lbCI7YToxOntzOjU6ImVtYWlsIjtpOjY1O319

Veeam is likely going to properly support XCP-ng.

And for what it's worth, you can use agent based Veeam backups in the VMs and that works fine.

@jebrown OK good to know. This may be different on different OSes though, Windows may just report actually used RAM and leave out anything that is cached.

@fred974 This is normal behavior, most modern OSes will use up as much RAM as they possibly can for caching, unused RAM is wasted RAM.

I'd adjust your alert settings if you're getting high RAM usage alerts. You also could change things in the OS to prevent as much RAM being used for caching, but I'd avoid that if possible.

The OS reports, via the tools, to the hypervisor about it's RAM usage, so if the OS is caching a lot, and therefore using most of it's RAM, it will report that to the hypervisor.

I think, could be wrong, Windows doesn't report cached RAM via the tools so it may show "real" RAM usage, but this is OS specific stuff and not really XCP-ng or any hypervisor specific stuff.

@plaidypus Ah gotcha, this makes sense.

I second scaling out instead of up.

If you're getting new hosts, I'd also keep in mind newer CPUs do have much higher per core performance (not sure what your current stuff is), so you also might be able to get away with less vCPUs and lower likelihood of NUMA spanning.

Either way though I think scaling out is the better direction to go.

@plaidypus High CPU usage doesn't necessarily mean that NUMA spanning will be much of an issue, really comes down to latency at that point. I'd say you should be OK to just go with it, I get the hesitation though.

@jebrown Yeah I just meant in general, not specific to you. I think most businesses aren't looking for this kind of specific workload so the demand isn't very high for it.

Are you currently using Intel GPUs with Flex for this use case? Or NVidia right now and just looking to change? It might be worth considering leaving VDI, not sure if you're in the position to be able to argue that, but there are often better solutions now.

Either way, I would personally also love to see this, I just think there are other things that more companies are asking for from XCP-ng right now.

@plaidypus I think the real question you should ask yourself is: do your workloads actually need to worry about the extra latency from a NUMA node span?

If you're not doing something pretty darn extreme here, I don't think it really will matter. There has been lots of talk for decades about this on VMware, but I just don't think it's that relevant anymore. Latency between sockets has gotten pretty good, so unless it's some really special workload, I don't think you should worry about this much.

The best way to validate is to just test and see how things go.

But do you have info on what workload this VM will be running?

@JamesG There probably is interest from homelab people, but for use in production setups, I don't really see a lot of businesses needing it.

VDI isn't really used that much by businesses now (at least the ones that do use it are slowly moving off it) and most that do use it don't need GPU acceleration. Often times it's for pretty basic applications, medical record systems, etc...

So I think that is where the lack of demand is coming from.

@R2rho Yeah that is really surprising.

I suppose it could be some kind of wider hardware incompatibility or something, but still crazy either way.

Glad you got that somewhat sorted out though.

@olivierlambert Yup, I've had exactly that a few times, usually on used boards.

@R2rho if possible, however annoying, I would also take the CPU out and check for pins on the motherboard being bent with a flashlight.

@olivierlambert Yeah @R2rho I am with this, it's strange to see memtest errors at all.

May be another component causing the failures though, and not the RAM itself. Possibly the board or the mem controller on the CPU.

You don't by chance have another AM4 CPU you can swap in do you?

Yeah wish I had a better response here but this is indeed odd.

Do you by chance have a PCIe ethernet card you can swap in to use for connectivity (and just not use the X550 ports), just to test and see if the X550 is causing the crashes.

It's a longshot though if I'm honest.

@Mefosheez I would try it on that other host you have and see if you run into the same issues, just for good measure.

Are they managed by the same XO? Maybe you can just migrate it to the other host so it's the exact same VM.

@Mefosheez Hmmm ok, yeah I would check a few of the logs on XCP-ng itself. https://docs.xcp-ng.org/troubleshooting/log-files/

Also, has this host been rebooted fairly recently?

@Mefosheez Hmmm lets double check all the advanced settings then, if this doesn't do it the might be worth checking the logs.

I'll give a reference point for an Ubuntu VM I have running, this is on an AMD Threadripper 1920X for what it's worth.

CPU Mask: none
CPU Weight: Default
CPU Cap: Default
Citrix PV drivers: disabled
HA: disabled
Affinity: none
GPUs: none
NIC: Realtek RTL8139
VGA: enabled
Video RAM: 8 MiB (might be worth bumping this to 16MiB to test, IIRC I had some VMs dislike 8MiB, don't recall the specifics)
Boot Firmware: UEFI
Secure Boot: enabled (probably leave disabled for now)
Viridian: disabled
CPU Limits: 4/4
Topology: Default
Memory is 2GiB

No VUSBs and no PCIs attached. Misc is all left default.

@Mefosheez Do you have secure boot enabled on these?

I haven't had any issues with NUMA node balancing, do you need this for a specific high performance application? Generally speaking it should "just work".

There are some things you can do to optimize but I'd only go down that road if you are running into issues with very wide VMs.

Also, are we talking cross socket NUMA nodes (e.g. multiple CPUs) or just nodes within an EPYC CPU?

@R2rho Do you have a way to view the console output when this happens? Curious if you had a display attached, you may see the remnants of the crash.

And I presume you can't get ping responses from it right? The other thought is maybe it's lost network connectivity but isn't actually a fully locked up host.

There is also info here on the log files you can check.

https://docs.xcp-ng.org/troubleshooting/log-files/