Why does emergencyShutdown take a lot longer than shutting down the host from the console?

[CodeMercenary]

This weekend I was working on my servers and I needed to shut them down. I closed all the VMs, then from the physical console I used the UI to tell the host to shut down. It shut down in about 30 to 60 seconds and powered off.

In testing my NUT scripts, they shut down all the VMs then issue the host.emergencyShutdown command and it takes the hosts about 8 minutes to shut down.

Any reason for that difference? Is there a command I can issue through the xo-cli that would cause the faster shutdown?

Another advantage is that the shutdown command from the console actually turns the power off while emergencyShutdown shuts everything down but doesn't power down the hardware, at least it never has for me.

[olivierlambert]

Do you have one VM which is acting as a SR for the rest?

[CodeMercenary]

@olivierlambert No, I don't use any VM as an SR in XO. Other than the local storage SRs, DVD, removable media and such, the only SRs are hosted on a Synology array or on my UNRAID server.

[olivierlambert]

IIRC, emergency shutdown will suspend all the VM, so it takes time to "save" the RAM on all VMs in the relevant SR. This means your VM won't lose their context (like when you hibernate on your laptop), at the price of some time needed to write the data on the disk.

If you just need to shutdown the host without doing suspend on VMs, you can use a regular shutdown via XO API.

[CodeMercenary]

@olivierlambert I understood what emergencyShutdownHost does, I was surprised that it seems to take a long time even if all the VMs were stopped before executing it. There should be nothing to suspend but it still takes about 8 minutes for the server to finish the shutdown.

I will start using host.stop instead of host.emergencyShutdownHost for the hosts that have no running VMs. Realistically, when having NUT shut it down, I'd rather the host just issue a clean shutdown command to any running VMs. I'm not sure what host.stop will do if there is a running VM. If it would politely ask it to stop then that would be perfect, if it yanks the virtual power cord then I wouldn't like that.

The ideal is that no host has a running VM by the time I want to shut it down but since NUT runs in a VM at the moment, one host will have that one running VM. I'll be in a bit of a race condition if I issue host.stop immediately followed by shutdown now. It's virtual murder-suicide but the murderer's life depends on the murdered. Can Linux shut down before xcp kills it? Might be an interesting test.

[manilx]

@CodeMercenary If the host.stop command does the same as a "reboot" or "shutdown" on the host cli than it just shuts down the VM's (no power plug pull) and then reboots or shuts down.

Do this all the time after a yum update...

Would be nice if you could share your nut setup to the community

[CodeMercenary]

@manilx That is great to hear, thank you.

I will definitely share it with the community. Still trying to iron out some of the wrinkles. Testing requires me to let all my servers get shut down so I'm limited in how frequently I can test the solution.

This weekend was my third pull-the-plug test and it was the closest to totally working. In fact, I think this test did totally work, but due to network changes I had to shut down xcp because I've found it gets really mad if you change anything about its network while it's running. It was using the physical console to shut it down that I was shocked at how fast it shut down. That's why I posted to ask about the difference.

I think changing my script to just use host.stop will resolve my last concerns about the script. Having a faster shutdown for xcp might also allow me to go back to my original design when I let the more important VMs live a bit longer. I originally staged when VMs got shut down so the important once could survive a 5 or 10 minute power outage. Turns out with xcp taking 8 minutes to shutdown after the VMs were down, I had to change my script to start to close everything as soon as it was clear that this wasn't just a small power blip.

When I post it though, everyone will need to recognize that I'm no bash coder. I write code but this is the only bash stuff I've done so it could be rough.

[CodeMercenary]

@manilx For host.stop, what does the bypassBackupCheck do? I think bypassEvacuate is pretty clear, that must mean "don't try to migrate running VMs to another server before stopping." I feel like I would want to bypass whatever the backup check is because I've lost power and the server needs to be shut down.

[manilx]

@CodeMercenary Can't help you there. The only thing I use(d) is the cli on the xcp hosts themselves.
reboot or shutdown