XOA Failing on Check-in

acomav

Hi,
I have also started having this issue.

My error:

✖ 15/16 - Internet connectivity: AggregateError [ETIMEDOUT]: 
    at internalConnectMultiple (node:net:1118:18)
    at internalConnectMultiple (node:net:1186:5)
    at Timeout.internalConnectMultipleTimeout (node:net:1712:5)
    at listOnTimeout (node:internal/timers:583:11)
    at process.processTimers (node:internal/timers:519:7) {
  code: 'ETIMEDOUT',
  url: 'http://xen-orchestra.com/',
  [errors]: [
    Error: connect ETIMEDOUT 185.78.159.93:80
        at createConnectionError (node:net:1648:14)
        at Timeout.internalConnectMultipleTimeout (node:net:1707:38)
        at listOnTimeout (node:internal/timers:583:11)
        at process.processTimers (node:internal/timers:519:7) {
      errno: -110,
      code: 'ETIMEDOUT',
      syscall: 'connect',
      address: '185.78.159.93',
      port: 80
    },
    Error: connect ENETUNREACH 2a01:240:ab08::4:80 - Local (:::0)
        at internalConnectMultiple (node:net:1182:16)
        at Timeout.internalConnectMultipleTimeout (node:net:1712:5)
        at listOnTimeout (node:internal/timers:583:11)
        at process.processTimers (node:internal/timers:519:7) {
      errno: -101,
      code: 'ENETUNREACH',
      syscall: 'connect',
      address: '2a01:240:ab08::4',
      port: 80
    }
  ]
}

I have two XOA appliance running in different locations. One works fine but the XOA version is: 5.95.1
The one that has started failing is running the latest version: 5.100.2

Traceroutes from the working XOA get to: (I'm in Australia hence the long response times)

...
16 prs-b1-link.ip.twelve99.net (62.115.125.167) 282.574 ms 282.700 ms freeprosas-ic-367227.ip.twelve99-cust.net (80.239.167.129) 303.985 ms
17 freeprosas-ic-367227.ip.twelve99-cust.net (80.239.167.129) 302.850 ms 302.835 ms be1.er02.lyo03.jaguar-network.net (85.31.194.151) 309.182 ms
18 cpe-et008453.cust.jaguar-network.net (85.31.197.135) 310.999 ms be1.er02.lyo03.jaguar-network.net (85.31.194.151) 308.157 ms 308.477 ms
19 * cpe-et008453.cust.jaguar-network.net (85.31.197.135) 318.785 ms 309.982 ms

From the non-working XOA:

...
10 * be803.lsr01.prth.wa.vocus.network (103.1.76.147) 106.498 ms be803.lsr01.stpk.wa.vocus.network (103.1.76.145) 109.750 ms
11 * * *
12 * * *
13 * * *
14 mei-b5-link.ip.twelve99.net (62.115.134.228) 244.552 ms mei-b5-link.ip.twelve99.net (62.115.113.2) 243.988 ms mei-b5-link.ip.twelve99.net (62.115.124.123) 258.259 ms
15 freeprosas-ic-373578.ip.twelve99-cust.net (62.115.35.93) 256.427 ms * *
16 be1.er02.lyo03.jaguar-network.net (85.31.194.151) 279.685 ms 276.070 ms *

On the new XOA, I can manually telnet to 185.78.159.93 on port 80 and get a response so I am at a loss.
It is not affecting day to day work.
I was going to download the latest version of the XOA appliance and import my config and see if that does the trick......unless anyone here has any other tests to run?

acomav

Replying to myself here for an update.

I reinstalled the XOA appliance and imported my config. (On a different host in a different pool)
That took me back to XOA v5.98.1. Internet connectivity was fine.
I stayed on the Stable Channel and went up to 5.99.1. Internet connectivity was fine.

I have a Pool issue where the XOA was and I can't fix it until tonight. (I need to upgrade and reboot the master.)

acomav

@acomav Replying to myself again. After working for a few days, the issue restarted. I'll raise a ticket.

turnt

@acomav Same issue here. Upgraded to 5.100.2 and this has appeared. Like you, doesn't appear to be affecting anything and I changed nothing on any of my systems/network for this to appear, the only change was the XOA upgrade.

DustinB

Well then I guess I'm glad to say I'm not insane, I was thinking maybe it was network latency or something else (which is completely plausible) but as far as I could tell this seems like XOA is unable to ping the domain.

Which doesn't make any sense, it's not like the website has gone offline..

acomav

I was able to fix it in mine by disabling IPv6. (Which we don't run).

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
In order to verify that IPv6 is disabled, run:

cat /proc/sys/net/ipv6/conf/all/disable_ipv6
If the output is 1, we can say IPv6 is in disable state.

This is a temp fix until next reboot. Read here for a permanent solution:

https://bobcares.com/blog/debian-12-disable-ipv6/

After disabling IPv6, 'xoa check' immediately started working.

olivierlambert

So it could be a peering issue in IPv6. On my side, no problem, where are you based?

acomav

@olivierlambert

I am in Australia.

Danp

Isn't this the same issue caused by the Node upgrade? https://github.com/nodejs/node/issues/54359

notr1ch created this issue in nodejs/node

closed Happy eyeballs implementation times out prematurely #54359

acomav

@Danp Interesting. That will be it. Thanks for linking this.
In the mean time, I've put in a request with the Australian government to move us closer to Europe.