Unable to enable High Availability - INTERNAL_ERROR(Not_found)

olivierlambert

@psafont I'm not sure to follow, I don't remember seeing any documented endpoint related to prepare HA

psafont

@olivierlambert The call is indeed hidden from the docs, and only callable from inside a pool... it's called as part as Pool.enable_ha

olivierlambert

So we probably need to tell XO team the "right way" to enable HA because there's no way to know from "outside"

psafont

@olivierlambert

So we probably need to tell XO team the "right way" to enable HA because there's no way to know from "outside it's not meant to, xapi makes the call automatically.

I don't think so, it's xapi's responsibility to make that call

jmannik

@psafont
[22:13 vmhost13 ~]# xe pool-param-get uuid=213186d2-e3ba-154f-d371-4122388deb83 param-name=master | xargs -I _ xe host-param-get uuid=_ param-name=address
192.168.10.13
[22:13 vmhost13 ~]# cat /etc/xensource/pool.conf
master[22:14 vmhost13 ~]#

psafont

@jmannik Could you collect the file contents of /etc/xensource/pool.conf from all the other hosts? The command is failing in one of them, not on the master host.

jmannik

[22:27 vmhost12 ~]# xe pool-param-get uuid=213186d2-e3ba-154f-d371-4122388deb83 param-name=master | xargs -I _ xe host-param-get uuid=_ param-name=address
192.168.10.13
[22:27 vmhost12 ~]# cat /etc/xensource/pool.conf
slave:192.168.30.13[22:27 vmhost12 ~]#

[22:27 vmhost11 ~]# xe pool-param-get uuid=213186d2-e3ba-154f-d371-4122388deb83  param-name=master | xargs -I _ xe host-param-get uuid=_ param-name=address
192.168.10.13
[22:28 vmhost11 ~]# cat /etc/xensource/pool.conf
slave:192.168.30.13[22:28 vmhost11 ~]#

I think I see where the issue is, not sure how to solve it though

psafont

@jmannik The IPs match, and now I don't have an explanation on why is this happening, I'll take another look at the codepath, but that'll have to take a while, as work is piling up

jmannik

Ok, so in this process I have come across a re-occurring issue I have had with XCP-NG where it will have the wrong order for the ethernet interfaces.
Each of my hosts has a 1gbit interface onboard, then a 4 port 10gbit card
It SHOULD be ordering the interfaces like so:
ETH0 1gbit
ETH1 10gbit
ETH2 10gbit
ETH3 10gbit
ETH4 10gbit

But it will randomly decide upon install (VMHost11 was recently rebuilt due to an id10t pebkac issue) to order them like below for no apparent reason:

ETH0 10gbit
ETH1 1gbit
ETH2 10gbit
ETH3 10gbit
ETH4 10gbit

And to be able to re-order the interfaces its just a lot more difficult that I think it should be.

jmannik

@psafont said in Unable to enable High Availability - INTERNAL_ERROR(Not_found):

@jmannik The IPs match, and now I don't have an explanation on why is this happening, I'll take another look at the codepath, but that'll have to take a while, as work is piling up

Ahh but they dont match.
VMHost13 lists 192.168.10.13
VMHost12 and VMHost11 list 192.168.30.13

psafont

@jmannik ah, indeed. Do you know which server / interface holds the IP 192.168.30.13? I suspect is still VMHost13, but a different interface.

Until the members have configured their master as 192.168.30.13, you'll have this error. This can be done by a call, but since it's a delicate operation, it's better if there are no operations running on the pool. SSH into the VMHost13, and run

xe host-list name-label=VMHost13 --minimal | xargs -I _ xe pool-designate-new-master host-uuid=_

This should write the new IP to the files of all the pool members and stop blocking this issue from enabling HA

jmannik

@psafont Would designating a new pool master do the same thing?
I ran the above command and its had no effect

jmannik

said in Unable to enable High Availability - INTERNAL_ERROR(Not_found):

@psafont Would designating a new pool master do the same thing?
I ran the above command and its had no effect

Well, I tried changing the pool master and when VMHost11 was the master I was able to enable HA.
Switching back to VMHost13 as the master now so will see how that goes

jmannik

said in Unable to enable High Availability - INTERNAL_ERROR(Not_found):

said in Unable to enable High Availability - INTERNAL_ERROR(Not_found):

@psafont Would designating a new pool master do the same thing?
I ran the above command and its had no effect

Well, I tried changing the pool master and when VMHost11 was the master I was able to enable HA.
Switching back to VMHost13 as the master now so will see how that goes

Everything is working as expected/hoped.

So for anyone reading through this and wants a TL;DR

Issue was related to the pool master setting, changing the pool master to a different host and then back to the original fixed the incorrect settings allowing HA to be enabled