Unable to MIgrate VDI when host is low on free memory
-
I have a small server that I use as a test host, and I added some SRs on it that are network-based. One is NFS and the other is iSCSI. VMs work fine on these SRs, but when I go to migrate a VDI on a running VM, with the Xen tools, I get a "not enough free RAM" error regardless of whether I try to migrate it to the NFS or iSCSI SRs. This host is very short on RAM. I am running the latest patched 8.3 with QCOW enabled. I tried two machines. One was an existing with a VHD disk and the other is a VM with a QCOW2 disk.
Thank you for the help.
-
Hi,
Low how? can you provide some numbers? Thanks!
-
@olivierlambert Here is the amount of free RAM on the system.
But when I try to migrate a VDI to a new storage repo, I get this error:
HOST_NOT_ENOUGH_FREE_MEMORY(8663334912, 3104964608)
I can create a new VM on that SR without issue.
-
Do you have dynamic memory set for your VM?
-
You say you're trying to migrate a vdi, where are you trying to migrate it?
To a SR attached to your current host or to another host/pool?If you're trying to migrate it to another host/pool this is something that i've seen as well when dom0 is not assigned enough RAM, we usually gave them 16Gg which seemed to resolved these kind of issues back when we were running 8.2.0.
-
@olivierlambert I do not.
-
@nikade I am trying to move the VDI from one storage SR to another on the same host. This is not in a cluster. This is a standalone host.
-
@hitechhillbilly said in Unable to MIgrate VDI when host is low on free memory:
@nikade I am trying to move the VDI from one storage SR to another on the same host. This is not in a cluster. This is a standalone host.
Alright, check the /var/log/SMlog as well as /var/log/xensource to see whats going on, you'll find an error there.
-
@hitechhillbilly I'd say this is normal if you are low on RAM and you are doing a live VDI migration. XCP-ng requires some amount of free RAM on the host to be able to live migrate the VDI. The larger the VDI, the more RAM is needed but exact sizing is unknown to me. I've encountered this error numerous times so I consider it common.
The way around this is to shutdown the VM and then migrate the VDI. RAM requirements in that case are much much lower.
-
@bvitnik said in Unable to MIgrate VDI when host is low on free memory:
@hitechhillbilly I'd say this is normal if you are low on RAM and you are doing a live VDI migration. XCP-ng requires some amount of free RAM on the host to be able to live migrate the VDI. The larger the VDI, the more RAM is needed but exact sizing is unknown to me. I've encountered this error numerous times so I consider it common.
The way around this is to shutdown the VM and then migrate the VDI. RAM requirements in that case are much much lower.
Yea, I suggested giving dom0 16Gb. That was our standard sizing back in the days and it worked pretty well.
-
@nikade This is not related to Dom0 RAM. It's related to the host RAM.
-
@bvitnik said in Unable to MIgrate VDI when host is low on free memory:
@nikade This is not related to Dom0 RAM. It's related to the host RAM.
Could you please explain the difference between the term dom0 and host?
I havent been using xcp-ng professionaly for a while, but back when we were using it dom0 actually represented the "host" and it was also running all the management tools and services, hence why it some times needed a bit more ram.Not really sure if there's some kind of caching involved when it is handling the VHD, but we went from a 50/50 chance of succeeding with a vdi migration to something near 90% after giving dom0 16Gb ram. The only exception was bigger VDI's, those kept on failing.
-
@nikade said in Unable to MIgrate VDI when host is low on free memory:
Could you please explain the difference between the term dom0 and host?
This is what I mean:
Dom0 is not "the host". It contains a management layer for the host but in reality it is just another VM, highly specialized one but stil just a VM. It does not see host's RAM as it's own.
-
@bvitnik Thanks for explaining, I was always under the impression that dom0 was the "host" so to say.
-
@nikade You can even see that Dom0 VM has 16 vCPUs and host has 40 CPUs, in my case anyway. So Dom0 sees just some chunk of host resources.
-
@nikade It's because it's often a misnomer. There reason is simple: Dom0 is indeed the place where you SSH, having an almost regular Linux system (at first sight), the toolstack, backup export etc. So it seems indeed to be "the host".
However, in "reality", the host is the entire resources of the machine, all CPUs and memory. It includes the Dom0, but not only (Dom0 doesn't have access to all CPU and memory, "only" all physical devices): all other VMs.
The difference is subtle enough, especially for people not coming from Xen, "Dom0" doesn't give any clue about what it is ("Control Domain" is a bit better, even if "Domain" is still less clear than "Virtual Machine"). That's because when you boot, you do NOT boot Linux but a micro-kernel, Xen, that does have access to all the host. That's also what makes Xen design more secure than others: only a small kernel (200k LoC) has access to everything, unlike in KVM for example, where it's the full kernel with at least 20M LoC. But you cannot access "Xen" with SSH or anything, outside sending commands from the Dom0. Hence the confusion.
That's why I'm even myself, on regular basis, uses "host" instead of Dom0 in the sake of simplicity/clarity.
-
I just learned something new, thats awesome
