@itnok yea it appears the project uses the plugin api for Vagrant 1.x which may not work with Vagrant 2.x (the latest version) I'm not sure if the plugin api has changed significantly since I haven't written any plugins myself.
@r1 If it helps at all, I have seen this more often on the pool master than in other pool hosts. We are using XO delta backup on 125 VMs in this pool daily. So, the master is busy doing a lot of snapshot coalesce operations (lots of iSCSI storage IO) compared to other hosts. The other host that has hit 95% control domain memory use is also IO heavy (it has some database server VMs).
So, you can of course makes some config by hand to alleviate some of the cost of the architecture on virtualization.
But like you can imagine, the scheduler will move the vCPU around and sometimes break the L3 locality if it move it to a remote core.
I asked to someone more informed than me about that and he said that running a vCPU is always better than trying to make it run locally so it's only useful under specific condition (having enough resources).
You can use the cpupool functionality to isolate VM on a specific NUMA node.
But it's only interesting if you really want more performance since it's a manual process, and can be cumbersome.
You can also pin vCPU on a specific physical core to keep L3 locality, but it would only work if you have little amount of VM running on that particular core. So yes, it might be a little gain (or even a loss).
There is multiple ways to make the core pinned, most with xl but if you want it to stick between VM reboot you need to use xe. Especially since if you want to pin a VM to a node and need it's memory being allocated on that node, since it can only be done at boot time. Pinning vCPU after boot using xl can create problem if you pin it on a node and the VM memory is allocated on a another node.
You can see the VM NUMA memory information with the command xl debug-key u; xl dmesg.
Pin a CPU:
xl vcpu-pin <Domain> <vcpu id> <cpu id>
e.g. : xl vcpu-pin 1 all 2-5 to pin all the vCPU of the VM 1 to core 2 to 5.
xl cpupool-numa-split # Will create a cpupool by NUMA node
xl cpupool-migrate <VM> <Pool>
@tbluml I'm trying to make a MxGPU setup with similar hardware (dell r720, 2x E5-2650).
I got the same SR-IOV errors as you. I added the pci=realloc pci=assign-busses params.
Unfortunately the the system does not manage to boot when adding pci=assign-busses.
Root disk in not discovered and dracut shell is started.
Did you run into the same issue and if so how did you fix it?
If anyone else stumbles upon this.
I reinstalled on (usb) disk that is not connected to the raid controller and it seems to work now.
I speculate that since the controller is a PCI device and pci=assign-busses allows the kernel to override pci numbers the raid device cannot be found using the predetermined data in the initramfs. But that might be complete nonsense (no expert in these matters).
@olivierlambert Hey i have another question, If i run the VM on the second Server does it matter if the RAM is the same althought CPUs are the same? Make and Model of the RAM or can that cause a problem? Another question if i have Host 2 online and i run different VMs on each of the host meaning Mail and Web Server on one host and Cloud Storage Server on the other using the same iSCSI storage can this be done by adding a second host to XO? or could it be possible to combine both servers into 1 for more Memory?