Pool Master down/restart makes the whole pool invisible to XO till master is online again

gb.123

With reference to the above topic, shouldn't the other pool members be visible in XO (which has been installed from Source) ?

When I give reboot command to the Dom0 Master, the whole Pool becomes invisible to XO. (i.e. I am also not able to see other hosts which were not rebooted)
When the Master comes back online, the pool is visible again.

Version : XCP-NG 8.3 Alpha
XO Version : xo-server 5.109.0 / xo-web 5.111.0 (Created from Source)

Update : SSH directly to slave node works, but XO removes all the servers in the pool from Server list also. (The list is restored as soon as master comes back)

PS: This topic may be moved to Xen Orchestra since it seems to be an XO issue.

gskger

@gb-123 That is the intended behavior as far as I understood. Only the master has a R/W database for the pools metadata in a pool, so when the master is down, XO cannot manage the pool (change metadata, move VMs, ...). If you are restarting the master, you have to wait until the master is up again. If the master is broken, you can promote a slave to be the new master in an emergency transition.

olivierlambert

That's correct. It's a "by design" thing in XCP-ng/XenServer, it's not an XO limitation. You can only connect to the master, all slaves will only redirect to the master. That's the same thing for any XAPI client (even xe CLI!)

That's how it works. It's OK to lose connection during a master reboot. If it fails or never come back, you can elect a new master manually on any slave console.

If you want to do that done automatically, then you might want to test HA (but I discourage it, it's a lot more complexity for something happening at a very rare pace)

gb.123

Thank you sooo much for a quick reply !

For promotion, shouldn't XO actually show an option to promote the slave to master ? (I guess the GUI is not built yet)

If all the slaves disappear, then the only option would be to SSH into a slave and promote it using CLI. Isn't it ?

olivierlambert

And how XO could promote a slave if there's no XAPI connection anymore? If you lose the master, you can't do any administrative action on the API anymore, period.

If you lose the master (eg sudden power loss):

all VMs running on the slaves will continue to work
but no XAPI operation can take place (no snap, no VM shutdown, nothing)

If the master will come back, just wait. If it doesn't, SSH to a slave and promote it to master.

As you can see, it's OK and there's no problem at all to reboot a master for maintenance or updates, for example.

gb.123

You are right. I had a misconception as I thought when I add a 'Server' to Server Section in XO, it individually creates a connection to all Servers directly (since it takes username and password individually for each server)

Hence I thought it would still be able to communicate to slaves using the supplied password.

Which brings me to a question : If a particular VM is configured as HA and is running on the Master Node, How will it shift to a slave node when Master goes down ? Or am I missing something here ?

Am new to XCP and have only recently started testing this.

I must stay I am amazed by such a friendly community and a blazingly fast response !

olivierlambert

XO doesn't create a connection to each server. In Settings/server, you just need to add the master, and nothing else.
Any request on the pool will go to the master (even a xe vm-start on a slave will send the request to the master). A slave can't be reach directly from XAPI. However, each slave got a copy of the master database (each modification is replicated to all slaves).
If you have HA enabled and VMs also in HA mode, it will be detected that the master died, and an election will start to elect a new master automatically, and all the VMs with HA enable on the dead node will be restarted elsewhere.

However, HA can be tricky. You need to disable it during maintenance and update to avoid potential dangerous side effects. I wouldn't recommend to enable it except if you really know the platform. So it's not for beginners.

gb.123

@olivierlambert

Any possibility (practical benefit) of enabling HA for VMs (using Shared Storage) but not for the xcp-ng master ? will that work ? Or would I need to enable HA for XCP-NG pool master also ?

olivierlambert

HA is enabled at the pool level. Read this blog post I wrote 9 years ago but still relevant: https://xen-orchestra.com/blog/xenserver-and-vm-high-availability/