Can't start or shutdown VM's after storage was powered off
This post will be less narrow and specific than we would all like - But it is my situation.
I have :
- Pool of 3 XCP-ng v8.0. on 3 Dell T5500's. Recently converted from ESXi - 3 months ago. Approx 50 VM's.
- 2 separate FreeNAS storage computers.
I have used XenServer 6 for years in a business environment.
I Left HA and VM's running.
I Shutdown one FreeNAS box due to disk error to investigate.
Problem: (trio of servers)
After turning FreeNAS back on, I could not do much with Xcp-ng.
I can ssh.
I can now use the XCP-ng center.
Most VM's are starting, but not started.
I tried to shut down VM's using command line
xecommands per https://support.citrix.com/article/CTX220777
I can not shut most of them down, I can't start them.
I could not do much with the system.
I used commands "like" these and others.
96 xe pool-list 101 /etc/init.d/xapissl restart 104 service xapi status 105 service xapi restart 107 netstat -an | grep 172.16.2.30 111 cat network.conf 112 cat xapi-ssl.pem 113 hostname -f 116 xe host-list params=uuid,name-label,host-metrics-live 117 xe host-is-in-emergency-mode 121 xe host-emergency-ha-disable --force 122 xe sr-list 123 xe host-list params=uuid,name-label,host-metrics-live
Not being able to use any particular XCP-ng server, I powered them all off. Then I set up a 4th fresh XCP-ng on another computer - Dell R620.
Problem: (Dell R620)
- mounting exiting SR's takes much longer than before. Instead of 10 seconds, it's two minutes for example.
- The windows 2012 w/exchange 2013 VM takes hours to start and exchange services don't start. The network and sharing dialog won't load.
I realize this sounds like a soap opera, but it is my situation. It probably sounds like a country song, I lost my job, then my car, then my wife, then my.....
I will post the last error messages from the XCP-ng log later today.
Many thanks for your patience with me.
I edited the post above in many places. Read it again if you already read it.
There are no errors in the XCP-ng center log.
I clarified the post title and that the original three servers are powered off. The R620 is the only one running now.
S.Pam last edited by
Did you restart the entire pool? Sounds like you might have NFS issues like stale mounts lingering. This could make things very slow on the XCP-ng side. Do you export your NFS with fsid option? You should.
Where would the stale mounts be? .. in XCP-ng? or in Freenas?
S.Pam last edited by S.Pam
In FreeNAS. exports file you should have fsid=123, with a unique number for each exported path. If not it can cause a problem if the nfs server is rebooted while the pool is up.
The stale mounts are in xcp-ng.
But this is just a theory.
During the power outage. Did all xcp-ng hosts and the FreeNAS as boxes go down? How did you boot up? You wrote you set up fresh servers?