Best Way to Live Migrate XO VM
-
I'm having a live migration issue. I needed to migrate my XO VM to another XCP Host so I can reboot the Host it was on (master) to finalize updates install (using RUP). The XO VM got about 95% migrated and seems to have hung. I now cannot connect to XO Web UI. I tried to connect to both my XCP hosts via ssh to force a shutdown and power it back on, but unable to do so as the VM now shows on both XCP Hosts
What's the best way to migrate a XO VM from one XCP Host to another? It's an Ubuntu 24.04 VM and Linux guest tools is installed on it. Also, how can I fix my current XO VM hung situation? Using XO Lite didn't help either
Thank you. -
C coolsport00 marked this topic as a question
-
Hi,
If the migration doesn't work, it means there's something wrong happening in your VM (it doesn't cooperate). Check your memory settings to be sure dynamic min = dynamic max = static max.
Then, check from the OS perspective if there's enough free RAM and if the OS isn't frozen or something that could cause the migration to fail.
Live migration should work at 100% of the time when the guest cooperate, so there's clearly a problem.
-
@olivierlambert I have migrated other VMs just fine. Never had a problem. It kinda makes sense migrating the XO VM hiccups a little bit. My vCenter VM does the same for 30-40secs when I VMotion it in vCenter, but it fully recovers. My XO VM hasn't recovered in XO to this point. And to be clear..yes, my XO VM is on my XCP Hosts...not in VMW.
How do I do what you ask? I tried to SSH into the XO VM (and this is on sources in my lab, btw) and it won't connect. I don't remember how much RAM I gave it. I want to say 2 vCPU and 4GB RAM..maybe 8GB. I can log into XO-Lite, but there is no menu to view my XO VM resources. Ideas/thoughts?
Thanks Olivier -
@olivierlambert If I SSH onto the XCP Host it was orig on (XCP Host1) and run xe vm-list it shows on there and running; If I SSH onto XCP Host2 xe vm-list shows it on there and running too.
Also...the XO VM was fully updated; no outstanding commits. I just updated it earlier this morning. -
@olivierlambert And, below is the msg I get when I attempt to force shut the XO VM down directly on each XCP host:
xe vm-shutdown name-label=nkc-xo force=true
The server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem.
message: Object with type VM and id 95ad90cc-85b0-98c6-d81d-61ba82742947/config does not exist in xenopsd -
I got the XO VM back up by rebooting my Pool Master XCP Host which forced the VM to shutdown. Not thrilled this happened. Migrating the XO VM in XO needs work. So, for the process of rebooting XCP Hosts after applying patches, either via RUP or manually, use the following process (Vates Team correct as needed):
1. Apply patches
2. Migrate all VMs off the Master XCP Host. If the XO VM is on
this Host, do not migrate
3. Log out of XO
4. SSH into the XO VM and shut it down
5. Log into the Master XCP Host XO-Lite
6. SSH into the Master XCP Host and reboot it
7. Once the Master XCP Host is back online, power the XO
VM back on via XO-Lite
8. Migrate VMs back to Master XCP Host as needed
9. If Master role changed to another Host, from the Pool >
Advanced tab, change the Master back to desired XCP
Host
10. Reboot other XCP Hosts in Pool to finalize applying
patches. Before doing so, make sure to migrate VMs off -
Now that my XO VM is back up...I tried to live migrate some of my VMs in my Pool and migrations are failing (Linux & Windows), and I can no longer see the VM's console. They're basically "hung" it seems. And, I can't power them down. The same behavior has now happened to them as my XO VM. Live migrations did work ok before the issue I had with my XO VM.
-
Sorry for all the comments/posts...but I think I may have figured out why the failures occurred. I can again live migrate VMs after I shut down my XO VM then rebooted both of my XCP Hosts. Of course, I've not re-attempted live migrating my XO VM. Too risky! Again, that should be fixed.
One of the requirements of live migration is guest tools needs to be installed, correct? So, it appears..if an attempt is made to live migrate a VM which doesn't have tools installed, it "hangs". I have several VMs like that. They're software appliances and currently I'm prevented from installing tools on them. After the non-tools installed VM hangs attemping a migration, if you attempt to live migrate a VM which does have tools installed, it will also hang...and my guess is because of the failure of the 1st VM without tools installed. Not sure why. If that is the case, that needs to be fixed as well. If this is not accurate, I would like to know why this migration hang/failure behavior occurs. I can re-attempt to live migrating a VM which doesn't have tools installed...but don't want to go through the hassle of powering everything down and rebooting my Hosts to get it working again
