@gnpcx What did you end up doing for this? The docs dont seem to have any guidance.

Its not solving it, but you can run

echo 3 > /proc/sys/vm/drop_caches

to release some of the cache again, without interfering with running processes.

[root@host2 ~]# free -m
total used free shared buff/cache available
Mem: 15958 3308 158 8 12491 2355
Swap: 1023 177 846
[root@host2 ~]# echo 3 > /proc/sys/vm/drop_caches
[root@host2 ~]# free -m
total used free shared buff/cache available
Mem: 15958 3308 2598 10 10051 2751
Swap: 1023 177 846

Agreed, but the downside is to update the template from time to time. We'll probably create a new Packer plugin to work with XO API, so it will be easier to use and rely on 🙂 (Packer goal is to prep' a template from scratch and up to date for you, that you can use for solution 2. then)

@epretorious I would add that you have to be careful about overprovisioning when NUMA/vNUMA kicks in, that is when you allocate more VCPUs to exceed the number of physical CPUs of a bank of them as well as the associated physical memory (assume, for the sake of argument, you have two banks of physical CPUs and each has directly accessible to it one of two banks of memory) then things get inefficient because a CPU may need to go across to a different bank of memory to access data and there is additional overhead involved. See for example this article and the two preceding it:
https://blogs.mycugc.org/2019/04/30/a-tale-of-two-servers-part-3-the-influence-of-numa-cpus-and-sockets-cores-persocket-plus-other-vm-settings-on-apps-and-gpu-performance/

-=Tobias

@Gheppy

Thank you very much for the quick reply. I thought so indeed. Will this also be the case running Workstation using Linux?

@olivierlambert its looks like it was not from XCP but from the cloud provider managing it (Cloudstack). as soon as max restrictions were removed i could assign all 128 threads as vCPUs to a single VM.

Forget about static min.

This means: start with 1GiB (dynamic min is memory on boot basically). You can balloon up to 4GiB while running (by changing dynamic min to 4GiB), and go down again to 1GiB by set again dynamic min to 1GiB => dynamic is your "live" range.

You can change the dynamic max (while the VM is running) up to 5GiB.

Ballooning will change the amount dynamically (if needed) inside your 1GiB->4GiB range while live.

But frankly, stick with static memory in general (having dynamic min == dynamic max == static max), this will solve a lot of headaches.

Thank you all for this wonderful information.

@Andrew I’ve started to experiment with Duet + OpenCore chain loading but I'm getting the same results…

I’ve tested it now on two different Legacy Machines, so either I’m setting it up wrong or Duet is the issue all around the baord… why could Duet potentially not be happy?

It’s not my target system, it’s gotta be something to do with the file structure or the DuetPkg itself, I've also checked for Debug Logs from OpenCore, seems it ain't booting at all, so it's definitely an issue early on.

@tjkreidl BIOS

@olivierlambert

Unfortunately I've moved VMs around (this is all testing at the moment) however there were other VMs with static=dynamic=2Gb which I could move.

Could you please either explain or point me to the documentation where I can understand how your comment works, please?

Hmm I would also check for BIOS/firmware update

What's the purpose of this other VNC daemon on TCP/9500?

From the Xen Project wiki page on XCP:

The ports used for VNC console are 5901-5999, and the ports used for text console are 9501-9599.

AFAICT: TCP/5900 & TCP/9500 belong to Dom0...

# xenstore-ls /local/domain/0 control = "" feature-poweroff = "1" feature-reboot = "1" feature-suspend = "1" domid = "0" name = "Domain-0" vncterm-pid = "1276" console = "" vnc-port = "5900" tc-port = "9500" vm = "/vm/7a4488c6-d4ac-4aaf-83f0-fb6d3f557f48"

But I don't know how to connect to DomU #1's console because there is no entry in Xenstore for DomU #1:

# xenstore-ls /local/domain/1 xenstore-ls: xs_directory (/local/domain/1): No such file or directory

Ideas? Suggestions?

Eric P.
Reno, Nevada

You could maybe raise the memory a bit, but yeah, the software RAID thing might hurt more than anything else 🤔

Maybe switch to full soft RAID with mdadm, that might be better 🙂

My particular setup may be a bit weird but here's a better description. I have my primary pool of servers running at home, and XOA runs there. I have a site-to-site VPN between home and the remote location so I can reach the remote host which is also managed through my XOA instance through the VPN tunnel. I don't need any proxies for this to work because I setup a VPN tunnel and XOA and the remote host have direct IP connectivity.

On the remote host, I have a VM that runs in an isolated network not reachable from home, so I cannot connect directly to the guest with VNC. If I'm connected to my home network I can see this guest through XOA. If however I connect to the VPN service at the remote end (different than the site-to-site tunnel) to get access to normal resources on the remote end and also want to use my test VM it becomes difficult to reach the VM because I can't reach my XOA instance back at home easily.

I'm not sure a proxy would solve this. I followed the instructions to test XO Lite and that looks like it is sort of what I am looking for. When I log into my remote host I can see the dashboard and the list of guests, but I don't see the console for the guests. Is the console supposed to be present or is the plan for it to be exposed in the future? If so, that would solve my problem.

Ah great news then!