Here is the log for the GUI Backup function:

Here is my story:
I responded to the notice regarding the latest update of the main branch, Master, commit 9cdaf, by updating the XO-CE vm, rebooting the XO-CE vm, bringing all 3 XCP-ng hosts on line, and then running the XO-CE GUI and selecting Pool Rolling Update.
The rolling update cycled through all 3 hosts in the pool.
I then attempted to run a pool metadata and xo_config backup.
It failed by timing out at 5 minutes.
I decided to shutdown the pool master, so I transferred all VMs to the other 2 hosts and shutdown the master for 15 seconds. I powered on the master, and transferred the VMs back to the master.
I shutdown the other 2 hosts, and then powered them back on. All hosts are indicating "green" status.
At this point I attempted to run the pool metadata and xo_config backup, and it ran successfully.
I waited 2 minutes and ran it again, with success again!
Lastly, I powered down the other 2 hosts, and with the VMs all running on the master, I ran the pool metadata and xo_config backup again, with apparent success.
If someone can tell me the location of the logs that recorded this operation, I will look at them closely to confirm that the pool metadata and xo_config backups actually took place.
Thank you.

@christopher-petzel
Great news!
I will test and report after it is available in a global update.
Thank you Christopher.

@jb
@olivierlambert
Oliver, you may be right, wondering whether they really worked. How can I prove that the metadata and xo-config backups were truly successful?

Below, I pasted in the GUI backup log from XO5.

@jb
You have hit on something that ties into what I have discovered. Since July 1, on my XCP-ng / XO-CE cluster, the metadata and xo-config backup job only runs successfully when I have all 3 hosts in my cluster active, and after I have reset the tool stack on the Master host.

Because I am running the XCP-ng and XO-CE on an off-grid solar/battery system, independent of my electric bill, I try to limit total power to 250 watts for all devices (network switches, iSCSI NAS, XCP-ng hosts, etc). Consequently, I run only one host on a regular basis. When it is necessary to update XCP-ng, I turn on all hosts and do a rolling pool update. If I manually perform a metadata and xo-config backup while all of the hosts are on, only then will the backup be successful.

Before July 1, the metadata and xo-config back was always successful, whether it was scheduled or manual.

Hopefully, this will turn on a light bulb in the head of a Vates engineer, and he or she will solve this issue!

A little more information about my post...
5 minutes ago, I deleted the metadata and xo-config backup job.
Next, I deleted the xo-config-backups directory on the remote NFS.
I also deleted the xo-pool-metadata-backups directory on the remote NFS.
I then created a new "metadata_and_xo_config_backup" job and then ran it.
The new job created both a xo-config-backups directory with sub dirs, as well as a xo-pool-metadata-backups directory under the /Backups NFS share on the NAS.
The xo-config-backups directory has a subdir and it holds a data.json (size 22.55KB), and a metadata.json (size 233 Bytes).
The xo-pool-metadata-backups directory has a subdirectory structure that has no saved data, no files.
The job failed after waiting a 5-minute period.

Also seeing the "Body timeout error" only on the metadata and XO-config backups for the past 2 weeks. All the VMs are backing up just fine.
The Remote is NFS on a share called Backups:
NFS Backups \192.168.191.8:Port:/Backups Click to edit
2 TiB / 4.84 TiB 105.4 MiB/s / 1.8 GiB/s None

I recently needed to restore from the metadata and xo-config backs and was able to find an old one, but can't make any new ones, so it is a source of concern.

@acebmxer
I appreciate your help with this issue of mine.

From my own unpriviliged account, I have always run 'sudo' followed by whatever command was needed. This has worked for more than a year without issue, but last week it all fell apart!

Trying to remove and reinstall nodejs and nvm using sudo from my account did not work.

However, I did find a way to get nodejs updated to version 24 and ronivay's update xo-install.sh script worked just fine. Instead of using sudo followed by the xo-update.sh command, I typed sudo -i

That brought me to the root prompt in the /root directory. I then installed nvm, sourced it, and installed nodejs version 24.

Still as root, I changed directory to the location of the xo-install.sh script and ran it without error, and it updated the the latest version of XO.

I logged out as root and then logged back in using my unpriviliged account. I am now able to run ronivay's xo-install.sh script by using sudo ./xo-install.sh with my own account. I hope it continues to work fine.

In the meantime, I'll experiment using your install_xen_orchestra script on a new VM and let you know how it goes.

Thank you.

@acebmxer
As you can deduce, I already have a VM that is running XO-CE.
I'm guessing that my first move should be to shutdown that VM and create a new VM to use as the Xen Orchestra controller. That will be my next move. If I run into trouble, I can always turn off the new VM and power on the one I had been using.

@acebmxer
I am using ronivay's script, and didn't know about yours, but I will be happy to use it and see if it works at this point.

I will attempt to follow the instructions that I see on your github page and get back to you soon.

Thank you for responding to my post.

Here is the entire log, not very long, but it shows that nodejs and yarn are up to date.

$ cat /home/gsbrown/XenOrchestraInstallerUpdater/logs/xo-install.log-202604171616
+ command -v git
/usr/bin/git
+ cd . && git config --get remote.origin.url
https://github.com/ronivay/XenOrchestraInstallerUpdater.git
+ cd . && git status --porcelain
?? .yarn/
?? .yarnrc.yml
?? ls
?? nodejsv24-updater-for-24.04-LTS.sh
?? package.json
Running script version 188de7e with config:

PORT="443"
PROXY_PORT="443"
INSTALLDIR="/opt/xo"
SELFUPGRADE=true
CONFIGUPDATE=true
REPOSITORY="https://github.com/vatesfr/xen-orchestra"
BRANCH="master"
PLUGINS="all"
AUTOUPDATE="true"
OS_CHECK="true"
ARCH_CHECK="true"
PRESERVE="3"
PATH_TO_HTTPS_CERT=$INSTALLDIR/xo.crt
PATH_TO_HTTPS_KEY=$INSTALLDIR/xo.key

+ id -u
0
+ uname -m
x86_64
+ command -v xe
+ grep ^VERSION_ID /etc/os-release | cut -d'=' -f2 | grep -Eo '[0-9]{1,2}' | head -1
24
+ grep ^NAME /etc/os-release | cut -d'=' -f2 | sed 's/"//g' | awk '{print $1}'
Ubuntu
+ grep ^GRUB_DISTRIBUTOR /etc/default/grub | grep 'Xen Orchestra'
+ command -v dnf
+ command -v apt-get
/usr/bin/apt-get
+ command -v systemctl
/usr/bin/systemctl
+ openssl x509 -pubkey -noout -in /opt/xo/xo.crt | openssl md5
MD5(stdin)= ba9cd49e64a4b58a2f1e1a2b60784d60
+ openssl pkey -pubout -in /opt/xo/xo.key -outform PEM | openssl md5
MD5(stdin)= ba9cd49e64a4b58a2f1e1a2b60784d60
+ df -P -k '/opt' | tail -1 | awk '{print $4}'
5484968
+ grep MemTotal /proc/meminfo | awk '{print $2}'
12239640
+ node -v 2>/dev/null| grep -Eo '[0-9.]+' | cut -d'.' -f1
23
+ curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg -o /etc/apt/trusted.gpg.d/yarn.asc
+ curl -sL https://deb.nodesource.com/setup_24.x | bash -
2026-04-17 16:16:35 - Installing pre-requisites

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Hit:1 https://dl.yarnpkg.com/debian stable InRelease
Hit:2 https://deb.nodesource.com/node_24.x nodistro InRelease
Get:3 http://security.ubuntu.com/ubuntu noble-security InRelease [126 kB]
Hit:4 http://us.archive.ubuntu.com/ubuntu noble InRelease
Get:5 http://us.archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB]
Get:6 http://security.ubuntu.com/ubuntu noble-security/main amd64 Components [21.5 kB]
Get:7 http://us.archive.ubuntu.com/ubuntu noble-backports InRelease [126 kB]
Get:8 http://security.ubuntu.com/ubuntu noble-security/restricted amd64 Components [212 B]
Get:9 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Components [74.2 kB]
Get:10 http://security.ubuntu.com/ubuntu noble-security/multiverse amd64 Components [212 B]
Get:11 http://us.archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [1,901 kB]
Get:12 http://us.archive.ubuntu.com/ubuntu noble-updates/main amd64 Components [177 kB]
Get:13 http://us.archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Components [212 B]
Get:14 http://us.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [1,666 kB]
Get:15 http://us.archive.ubuntu.com/ubuntu noble-updates/universe amd64 Components [386 kB]
Get:16 http://us.archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Packages [32.5 kB]
Get:17 http://us.archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Components [940 B]
Get:18 http://us.archive.ubuntu.com/ubuntu noble-backports/main amd64 Components [7,340 B]
Get:19 http://us.archive.ubuntu.com/ubuntu noble-backports/restricted amd64 Components [216 B]
Get:20 http://us.archive.ubuntu.com/ubuntu noble-backports/universe amd64 Components [10.5 kB]
Get:21 http://us.archive.ubuntu.com/ubuntu noble-backports/multiverse amd64 Components [212 B]
Fetched 4,658 kB in 2s (2,921 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
2 packages can be upgraded. Run 'apt list --upgradable' to see them.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
apt-transport-https is already the newest version (2.8.3).
ca-certificates is already the newest version (20240203).
curl is already the newest version (8.5.0-2ubuntu10.8).
gnupg is already the newest version (2.4.4-2ubuntu17.4).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Hit:1 https://dl.yarnpkg.com/debian stable InRelease
Hit:2 https://deb.nodesource.com/node_24.x nodistro InRelease
Hit:3 http://us.archive.ubuntu.com/ubuntu noble InRelease
Hit:4 http://us.archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:5 http://security.ubuntu.com/ubuntu noble-security InRelease
Hit:6 http://us.archive.ubuntu.com/ubuntu noble-backports InRelease
Reading package lists...
Building dependency tree...
Reading state information...
2 packages can be upgraded. Run 'apt list --upgradable' to see them.
2026-04-17 16:16:43 - Repository configured successfully.
2026-04-17 16:16:43 - To install Node.js, run: apt install nodejs -y
2026-04-17 16:16:43 - You can use N|solid Runtime as a node.js alternative
2026-04-17 16:16:43 - To install N|solid Runtime, run: apt install nsolid -y

+ apt-get install -y nodejs
Reading package lists...
Building dependency tree...
Reading state information...
nodejs is already the newest version (24.14.1-1nodesource1).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
+ cd "/opt/xo/xo-src/xen-orchestra" && git remote set-url origin "https://github.com/vatesfr/xen-orchestra" &&             git fetch --prune &&             git reset --hard origin/master &&             git clean -xdff
From https://github.com/vatesfr/xen-orchestra
 - [deleted]             (none)                -> origin/backup-XO-Tasks
 - [deleted]             (none)                -> origin/mra-fix-remote-copy
 - [deleted]             (none)                -> origin/show_backup_merged_size
 - [deleted]             (none)                -> origin/test_replication
   6b1a2afc9..fbba05bbd  master                -> origin/master
 * [new branch]          doc-xo6-section       -> origin/doc-xo6-section
   b8b00733e..6c10859a0  encrypt_crendentials  -> origin/encrypt_crendentials
   48caca99c..b303e7f8c  feat/mcp-openapi-bootstrap -> origin/feat/mcp-openapi-bootstrap
 + b8eb4af6b...1feb8b695 feat/xo-server-ipmi-sensors -> origin/feat/xo-server-ipmi-sensors  (forced update)
 + 5d60a1dbb...184117ce1 fix_memory_leak_bis_final -> origin/fix_memory_leak_bis_final  (forced update)
 * [new branch]          hotfix/6.2.2          -> origin/hotfix/6.2.2
 * [new branch]          mra-acl-backup-jobs   -> origin/mra-acl-backup-jobs
   dc66c0e29..abdd95198  refactor_backup_clean -> origin/refactor_backup_clean
 + 31d488df2...21b79845b web-core/remove-uiObjectLink -> origin/web-core/remove-uiObjectLink  (forced update)
 + cf6268396...12ccb2e0a xo6/update-system-pages-layout -> origin/xo6/update-system-pages-layout  (forced update)
HEAD is now at fbba05bbd feat(backups): use XO Tasks (#9734)
+ rm -rf "/opt/xo/xo-builds/xen-orchestra-202604171616"
+ cp -r "/opt/xo/xo-src/xen-orchestra" "/opt/xo/xo-builds/xen-orchestra-202604171616"
+ cd /opt/xo/xo-builds/xen-orchestra-202604171616 && git rev-parse HEAD
fbba05bbde4514c63b2ddd97b5304de1210030e5
+ cd /opt/xo/xo-builds/xen-orchestra-202604171616 && git rev-parse --short HEAD
fbba05bbd
+ cd .
+ readlink -e /opt/xo/xo-server
/opt/xo/xo-builds/xen-orchestra-202604111527/packages/xo-server
+ cd /opt/xo/xo-server && git rev-parse HEAD
aa605639e46568061635695bb61492c0c990ead2
+ cd /opt/xo/xo-server && git rev-parse --short HEAD
aa605639e
+ cd .
Updating Xen Orchestra from 'aa605639e' to 'fbba05bbd'
+ cd /opt/xo/xo-builds/xen-orchestra-202604171616 && yarn --network-timeout 300000 && yarn --network-timeout 300000 build
yarn install v1.22.22
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
warning bare-fs@4.7.1: The engine "bare" appears to be invalid.
warning bare-os@3.8.7: The engine "bare" appears to be invalid.
error eslint-visitor-keys@5.0.1: The engine "node" is incompatible with this module. Expected version "^20.19.0 || ^22.13.0 || >=24". Got "23.11.0"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
+ rm -rf /opt/xo/xo-builds/xen-orchestra-202604171616

Just yesterday I attempted to update XO-CE from 'aa605639e' to 'fbba05bbd' and received a very similar error to @acebmxer. All subsiquent update attempts have failed, so I remain running on XO on version aa605639e.

I have been using the XenOrchestraInstallerUpdater by Roni Vay since April of 2025 weekly, and it has been very reliable. XO resides on a VM running Ubuntu 24.04.4 LTS \n \l which I apt update/upgrade weekly. There are 3 Intel-based hosts in the cluster (Intel(R) Core(TM) i7-10700 CPU) with 8 VMs and lots of memory.

I am hoping someone will see this and show me a way to fix it.

Thank you.

Below are the very last lines of the xo-install.log:

Updating Xen Orchestra from 'aa605639e' to 'fbba05bbd'

• cd /opt/xo/xo-builds/xen-orchestra-202604171616 && yarn --network-timeout 300000 && yarn --network-timeout 300000 build
  yarn install v1.22.22
  [1/5] Validating package.json...
  [2/5] Resolving packages...
  [3/5] Fetching packages...
  warning bare-fs@4.7.1: The engine "bare" appears to be invalid.
  warning bare-os@3.8.7: The engine "bare" appears to be invalid.
  error eslint-visitor-keys@5.0.1: The engine "node" is incompatible with this module. Expected version "^20.19.0 || ^22.13.0 || >=24". Got "23.11.0"
  error Found incompatible module.
  info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
• rm -rf /opt/xo/xo-builds/xen-orchestra-202604171616

@olivierlambert Thank you for your response.

I have followed most of the documentation on setting up HA, and made sure that I have 3 hosts. All are Dell 7090 64GB RAM with 2 dedicated HA NICS. XCP-ng 8.3 is patched, XO is up to date, Xen tools 9.4.0 on all Windows VMs, and using the Xen tools ISO that can be mounted.
eth0 1Gbs management network
eth1 10Gbs iSCSI network DAC
eth2 10Gbs VLAN Trunk, including backup network
eth3 1Gbs HA
eth4 1Gbs HA

Overall, XCP-ng and XO are doing an excellent job. I feel like I am on an Easter egg hunt when pouring through documentation, but that is mostly my problem, because I see that you have put a lot of effort into organizing and updating it.

My "use case" is replacing the VMware ESXi/vSphere 2-node cluster with XCP-ng and XO at the small high school (non-profit) where I volunteer as the IT guy. I feel that I am at the point where I trust XCP-ng and XO enough to make the switch during the summer break.

Sorry to run on. The yellow triangles behind the non-master hostnames are just a small nuisance. I do notice that the hosts are hard to shutdown. Sending a shutdown command from either XO or from the host console causes a host to reboot, but not shutdown, even after having previously entered maintenance mode. I feel that this is not a big issue, just another small nuisance. The XO cluster is maintaining the integrity of the guest VMs.

I will be reading all of the documentation I can, and upgrading to 8.4 when it is available. Thank you for supporting your awesome hypervisor.

@bradmbreer and @Andrew
Thank you for clearing up why I see the red triangle on all my hosts that are not masters.
I wonder whether the developers will hide this little blemish in their next version. It is unsettling to a recent XCP-ng/XO user like me.