Backup of Windows VM failing Health Check [Solved]

McHenry

How can the timeout be raised?

I have a Windows server VM that is slow to boot and eventually starts however the health check reports a timeout "timeout reached while waiting for OpaqueRef..."

olivierlambert

In the XO config file:

[backups]
healthCheckTimeout = '10m' # the default

McHenry

@olivierlambert

I still have the health check failing after 10 mins. Can I confirm the syntax. I have tried:

healthCheckTimeout = '15m'
healthCheckTimeout = '15 minutes'

olivierlambert

@McHenry can you try with the latest Citrix tools? It seems they fixed an issue on reporting the tool status.

McHenry

@olivierlambert

Have already upgraded and the health checks now work well The issue I have is our DR hardware is slow to boot some VMs during a health check so the 10 mins is not enough, i.e. large DB servers. Smaller servers boot quickly and the health check works.

olivierlambert

Oh sorry didn't scroll up enough this morning

We need to check internally if the timeout works as designed, let me ping @Bastien-Nollet or @stephane-m-dev

Bastien Nollet

@olivierlambert I'll take a look

Bastien Nollet

It appears that the value of healthCheckTimeout from the config file is indeed not taken into account.
We've created a card on our side to fix this issue, and we'll soon plan it with the team.

If you can't wait for the fix to be released, you can modify the default value of healthCheckTimeout in the code in files @xen-orchestra/backups/_runners/VmsRemote.mjs and @xen-orchestra/backups/_runners/VmsXapi.mjs, then restart XO server, and this should fix it until next update.

McHenry

@Bastien-Nollet

Thank you.

Bastien Nollet

Hi @McHenry

If you still have the problem, you can increase the healthCheckTimeout value as Olivier recommended (e.g. healthCheckTimeout = ‘30m’), however this value should be in the [backups.defaultSettings] section of the configuration file (or in the [backups.vm.defaultSettings] section) rather than in the [backups] section.

We've detailed the documentation a bit to make this more understandable: https://github.com/vatesfr/xen-orchestra/blob/a945888e63dd37227262eddc8a14850295fa303f/packages/xo-server/config.toml#L78-L79