A
Offline
Version 0.2.0 has been release - https://github.com/acebmxer/install_xen_orchestra/blob/main/CHANGELOG.md
Last changes has been about 1 month ago. I have been using it daily, mostly on Ubuntu server but have done some basic test on the other distros that it installs and you can log in with the default credentials. Just need some more feedback from other users, who would be willing to try the project out.
Also some other useful projects I have been working on
Not sure when this issue start but it is persistent on latest commit. I can either click between vms and/or refreshing the same page will allow the vdi to be show or not. Pictures show different vm's but that was jsut a quick way of getting the different visuals.
Can confirm cloud-init images does not allow you adjust disk size in v6 but does in v5 ui.
I think the main issue is on cloud-init images you can not select a pre-configured custom config templates to apply to the cloud-init image in v6.
This is true on v5 you have to select a config before you can incrase the disk size.
So if i set VMs to Best-effort or disabled for HA i get not enough memory.
@pkgw Do you have Twinstore configured and/or HA enabled? If no I will continue my issues at the twinstore post.
If i set to restart i get this....
host.setMaintenanceMode
{
"id": "35233210-4e37-4703-9bf6-9e8a9c24df9f",
"maintenance": true
}
{
"code": "HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN",
"params": [],
"call": {
"duration": 5,
"method": "host.disable",
"params": [
"* session id *",
"OpaqueRef:d9bd976f-445c-8244-1b56-879f446efc12"
]
},
"message": "HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN()",
"name": "XapiError",
"stack": "XapiError: HA_OPERATION_WOULD_BREAK_FAILOVER_PLAN()
at XapiError.wrap (file:///opt/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
at file:///opt/xen-orchestra/packages/xen-api/transports/json-rpc.mjs:38:21
at runNextTicks (node:internal/process/task_queues:65:5)
at processImmediate (node:internal/timers:472:9)"
}
No VM had HA set. All were set to disabled. Just set all running vms to best effort and still error.
host.setMaintenanceMode
{
"id": "35233210-4e37-4703-9bf6-9e8a9c24df9f",
"maintenance": true
}
{
"code": "HOST_NOT_ENOUGH_FREE_MEMORY",
"params": [
"OpaqueRef:9ada2e57-1c8a-acb8-3637-df6819b64aca"
],
"task": {
"uuid": "a73ffab2-ef34-7daa-8f26-0bb0c5769559",
"name_label": "Async.host.evacuate",
"name_description": "",
"allowed_operations": [],
"current_operations": {},
"created": "20260710T16:42:09Z",
"finished": "20260710T16:42:09Z",
"status": "failure",
"resident_on": "OpaqueRef:119546e4-e0fd-adc7-d42a-b17c4efe789d",
"progress": 1,
"type": "<none/>",
"result": "",
"error_info": [
"HOST_NOT_ENOUGH_FREE_MEMORY",
"OpaqueRef:9ada2e57-1c8a-acb8-3637-df6819b64aca"
],
"other_config": {},
"subtask_of": "OpaqueRef:NULL",
"subtasks": [],
"backtrace": "(((process xapi)(filename ocaml/xapi/xapi_host.ml)(line 629))((process xapi)(filename hashtbl.ml)(line 159))((process xapi)(filename hashtbl.ml)(line 165))((process xapi)(filename hashtbl.ml)(line 170))((process xapi)(filename ocaml/xapi/xapi_host.ml)(line 625))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/rbac.ml)(line 228))((process xapi)(filename ocaml/xapi/rbac.ml)(line 238))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))"
},
"message": "HOST_NOT_ENOUGH_FREE_MEMORY(OpaqueRef:9ada2e57-1c8a-acb8-3637-df6819b64aca)",
"name": "XapiError",
"stack": "XapiError: HOST_NOT_ENOUGH_FREE_MEMORY(OpaqueRef:9ada2e57-1c8a-acb8-3637-df6819b64aca)
at XapiError.wrap (file:///opt/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
at default (file:///opt/xen-orchestra/packages/xen-api/_getTaskResult.mjs:13:29)
at Xapi._addRecordToCache (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1078:24)
at file:///opt/xen-orchestra/packages/xen-api/index.mjs:1112:14
at Array.forEach (<anonymous>)
at Xapi._processEvents (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1102:12)
at Xapi._watchEvents (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1275:14)"
}
@acebmxer Do you have VMs that aren't configured for HA? Does it work if you temporarily disable HA?
yes disabling HA did work. Note HA was setup via Twinstore. I currently do not have a vm on that SR.
after manually migrate vms off host i was able to put host into maintenance mode to apply updates.
Edit after both host updated still not able to put host into maintenance mode...
host.setMaintenanceMode
{
"id": "c1372ec6-3651-4481-808b-34293e53e144",
"maintenance": true
}
{
"code": "HOST_NOT_ENOUGH_FREE_MEMORY",
"params": [
"OpaqueRef:9ada2e57-1c8a-acb8-3637-df6819b64aca"
],
"task": {
"uuid": "ad526091-5706-3e6b-a544-b7f84a5cfdb0",
"name_label": "Async.host.evacuate",
"name_description": "",
"allowed_operations": [],
"current_operations": {},
"created": "20260710T15:43:17Z",
"finished": "20260710T15:43:17Z",
"status": "failure",
"resident_on": "OpaqueRef:119546e4-e0fd-adc7-d42a-b17c4efe789d",
"progress": 1,
"type": "<none/>",
"result": "",
"error_info": [
"HOST_NOT_ENOUGH_FREE_MEMORY",
"OpaqueRef:9ada2e57-1c8a-acb8-3637-df6819b64aca"
],
"other_config": {},
"subtask_of": "OpaqueRef:NULL",
"subtasks": [],
"backtrace": "(((process xapi)(filename ocaml/xapi/xapi_host.ml)(line 629))((process xapi)(filename hashtbl.ml)(line 159))((process xapi)(filename hashtbl.ml)(line 165))((process xapi)(filename hashtbl.ml)(line 170))((process xapi)(filename ocaml/xapi/xapi_host.ml)(line 625))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/rbac.ml)(line 228))((process xapi)(filename ocaml/xapi/rbac.ml)(line 238))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))"
},
"message": "HOST_NOT_ENOUGH_FREE_MEMORY(OpaqueRef:9ada2e57-1c8a-acb8-3637-df6819b64aca)",
"name": "XapiError",
"stack": "XapiError: HOST_NOT_ENOUGH_FREE_MEMORY(OpaqueRef:9ada2e57-1c8a-acb8-3637-df6819b64aca)
at XapiError.wrap (file:///opt/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
at default (file:///opt/xen-orchestra/packages/xen-api/_getTaskResult.mjs:13:29)
at Xapi._addRecordToCache (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1078:24)
at file:///opt/xen-orchestra/packages/xen-api/index.mjs:1112:14
at Array.forEach (<anonymous>)
at Xapi._processEvents (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1102:12)
at Xapi._watchEvents (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1275:14)"
}
Possible error with Twinstore enabled HA?
Just tried to apply update. Unable to put master host into maintenance mode - not enough memory...
https://xcp-ng.org/forum/post/106814
Edit - Updated both host after manually migrating the vms. Also noticed when i thought i was updating the master host first it might not have been.
With Twinstore installed and enabled HA was also enabled. After first host came back on line i looked at the pool advanced settings and saw host 2 was master not host 1.
Both host have been updated and rebooted. @olivierlambert would this change post master from one host to the other? XO did loose access to the VMs while the first host rebooted. So i am alittle confused...
Once host 2 came back online pool master is switched back to host 1 again...
will look for any other odd things.
Think i am facing this issue now. Just tried to put master host into maintance mode to apply latest 8.3 updates and getting the error.
host.setMaintenanceMode
{
"id": "c1372ec6-3651-4481-808b-34293e53e144",
"maintenance": true
}
{
"code": "HOST_NOT_ENOUGH_FREE_MEMORY",
"params": [
"OpaqueRef:1571e2f7-c235-dc10-d766-8883ddd99fd0"
],
"task": {
"uuid": "631dc2d9-7598-8ce6-5dd0-558342eb7d1f",
"name_label": "Async.host.evacuate",
"name_description": "",
"allowed_operations": [],
"current_operations": {},
"created": "20260710T15:20:03Z",
"finished": "20260710T15:20:03Z",
"status": "failure",
"resident_on": "OpaqueRef:119546e4-e0fd-adc7-d42a-b17c4efe789d",
"progress": 1,
"type": "<none/>",
"result": "",
"error_info": [
"HOST_NOT_ENOUGH_FREE_MEMORY",
"OpaqueRef:1571e2f7-c235-dc10-d766-8883ddd99fd0"
],
"other_config": {},
"subtask_of": "OpaqueRef:NULL",
"subtasks": [],
"backtrace": "(((process xapi)(filename ocaml/xapi/xapi_host.ml)(line 629))((process xapi)(filename hashtbl.ml)(line 159))((process xapi)(filename hashtbl.ml)(line 165))((process xapi)(filename hashtbl.ml)(line 170))((process xapi)(filename ocaml/xapi/xapi_host.ml)(line 625))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/rbac.ml)(line 228))((process xapi)(filename ocaml/xapi/rbac.ml)(line 238))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))"
},
"message": "HOST_NOT_ENOUGH_FREE_MEMORY(OpaqueRef:1571e2f7-c235-dc10-d766-8883ddd99fd0)",
"name": "XapiError",
"stack": "XapiError: HOST_NOT_ENOUGH_FREE_MEMORY(OpaqueRef:1571e2f7-c235-dc10-d766-8883ddd99fd0)
at XapiError.wrap (file:///opt/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
at default (file:///opt/xen-orchestra/packages/xen-api/_getTaskResult.mjs:13:29)
at Xapi._addRecordToCache (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1078:24)
at file:///opt/xen-orchestra/packages/xen-api/index.mjs:1112:14
at Array.forEach (<anonymous>)
at Xapi._processEvents (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1102:12)
at Xapi._watchEvents (file:///opt/xen-orchestra/packages/xen-api/index.mjs:1275:14)"
}
Found root cause.... Forehead slap moment. NBD was not enable on the nics in the pool.
@poddingue Just ran one more backup to confirm working, and it fell back to full backup again....
What logs or other information should I look into to find the root cause or do you still think it fall under the known issue?
I re-enabled purge snapshot data when using CBT. And it did not fall back to full again either.....
2026-07-08T10_05_49.749Z - backup NG.txt
The unknow snapshot files are still pressent.
After removing the CBT enabled option on the actual vms and ran a delta backup it did not fall back to full.
2026-07-08T09_59_01.627Z - backup NG.txt
These are still here....
And there is this message about those files.
while looking into this further i thought I might be facing disk images showing up a snapshots because i saw this when i looked into the storage. Just an update on what may have cause my fall back to full backups if its releated. I migrated my 2 host from AMD to Intel I kinda did it a sloppy way. What I mean by that, is I think I migrated them between two pools more then need as i had 3 host already setup as intel. In the end they made their way back to the original SR after AMD host were removed from the picture. The first backup after migration was I believe a Full backup as it was over the weekend.
After expanding I saw my actual vms still there... I think those are from the current backups? since i truned off remove snapshot when using cbt as that used to fix this issue. But maybe it hasnt as the vdi's still cbt enabled?
So not sure why this problem has come back but all backups are failing back to Full backups.
2026-07-06T02_09_34.032Z - backup NG.txt <- this was a Planned full backup.
2026-07-07T02_00_00.007Z - backup NG.txt - first Delta backup after full fell back to full.
I just tried to disable "Purge snapshot data when using CBT" but still fell back to full. Unless i need to run a nother full job first?
Added local user support.
Current sample config file.
# Example xo-apply configuration.
# Copy this into your own PRIVATE repo, edit, then:
# export XO_URL=https://xo.example.lan XO_TOKEN=...
# xo-apply diff config.yaml
# xo-apply apply config.yaml
#
# Secrets never go in this file: use ${env:VAR_NAME} placeholders,
# resolved from environment variables when the tool runs.
#
# A section that is ABSENT is unmanaged (xo-apply won't touch or report that
# resource type). A present-but-empty section means "manage this type, none
# should exist" (only deleted when you pass --prune).
remotes:
# NFS share on a NAS
- name: nas-backups
type: nfs
host: 192.168.1.50
path: /export/xo-backups
# port: 2049 # optional
# mountOptions: vers=4 # optional mount(8) options
# SMB / Windows share — host is "HOST\share" (single backslash in YAML
# double-quoted strings must be written as \\)
- name: windows-share
type: smb
host: "192.168.1.60\\backups"
domain: WORKGROUP
username: backup
password: ${env:SMB_BACKUP_PASSWORD}
# path: xo # optional subfolder inside the share
# S3-compatible object storage (AWS, MinIO, Backblaze B2, ...)
- name: offsite-s3
type: s3
host: s3.us-east-1.amazonaws.com
path: my-bucket/xo-backups # bucket/directory
accessKey: AKIAEXAMPLE
secretKey: ${env:S3_SECRET_KEY}
region: us-east-1
# protocol: http # for http-only endpoints (e.g. local MinIO)
# Directory local to the XO VM (e.g. a mounted USB disk)
- name: local-disk
type: local
path: /mnt/backup-disk
backupJobs:
# Delta (incremental) backup of every VM tagged "critical", every night
- name: nightly-critical
mode: delta
vms:
tag: critical
remotes: [nas-backups, offsite-s3]
settings: # optional global job settings, passed through to XO
concurrency: 2
# timezone: America/New_York
# maxExportRate: 104857600 # bytes/s
# nRetriesVmBackupFailures: 2
# reportWhen: failure
schedules:
- name: nightly
cron: "0 2 * * *"
retention: 14 # backups kept on the remotes
# snapshotRetention: 3 # snapshots kept on the pool
# timezone: America/New_York
# enabled: false # schedules are enabled by default
# Weekly full backup of specific VMs, selected by name
- name: weekly-full
mode: full
compression: zstd
vms:
names: [dc-01, mail-01]
# ...or select by uuid: uuids: [770aa52a-fd42-8faf-f167-8c5c4a237cac]
# ...or pass a raw XO smart-mode pattern for anything more complex:
# raw:
# type: VM
# tags:
# __or: [[prod]]
remotes: [nas-backups]
schedules:
- name: weekly
cron: "0 3 * * 0"
retention: 8
# Disaster Recovery / Continuous Replication: instead of (or in addition to)
# remotes, target one or more SRs. mode:full => DR, mode:delta => CR.
- name: dr-critical
mode: full # delta = Continuous Replication
vms:
tag: critical
srs: [4991d4aa-ed84-599b-7d19-97f2f943a366] # target SR UUID(s)
# remotes: [] # SR-only is fine; may be combined with remotes
schedules:
- name: hourly-dr
cron: "0 * * * *"
retention: 3 # replicas kept on the SR
# Metadata backups: pool metadata and/or XO's own configuration.
metadataBackups:
- name: xo-config
xoMetadata: true # back up XO's own config
pools: [939ed551-fbd6-9868-52d8-d3997b7bf7da] # pool UUID(s) for pool metadata
remotes: [nas-backups]
schedules:
- name: daily
cron: "0 21 * * *"
xoRetention: 7 # XO metadata backups kept
poolRetention: 7 # pool metadata backups kept
# Mirror backups: copy an existing remote's backups onto other remote(s),
# e.g. push local backups offsite to S3.
mirrorBackups:
- name: offsite-mirror
mode: full # full or delta, to match the source backups
sourceRemote: nas-backups
remotes: [offsite-s3]
schedules:
- name: nightly-mirror
cron: "0 5 * * *"
retention: 14
# Sequences: run backup schedules one after another. Each step names a job and
# one of its schedules (from any job kind above, or already in XO). The sequence
# has its own cron for when the whole chain runs.
sequences:
- name: nightly-then-metadata
steps:
- { job: nightly-critical, schedule: nightly }
- { job: xo-config, schedule: daily }
cron: "0 22 * * *"
# enabled: false # sequences are enabled by default
# timezone: America/New_York
# Local users (XO's internal auth provider). Users created by an external auth
# plugin (LDAP/SAML/GitHub) are NOT managed here and are never pruned.
#
# Passwords are write-only: XO never returns them, so a real password can't be
# exported or diffed — but XO REQUIRES a password to create a user. So `export`
# writes the placeholder `password: ChangeMe` for every user.
# ⚠️ CHANGE these before importing into a real XO (or use a ${env:...} ref),
# otherwise every new user is created with the password "ChangeMe".
# The password is only used when a user is CREATED; for an existing user it is
# ignored (change it in the XO UI). Only `permission` is compared for drift.
users:
- email: ops@example.com
password: ChangeMe # required by XO; change before import
permission: admin # none | read | write | admin (default: none)
# Local groups. Members are referenced by email and resolved to ids at apply
# time; each member must be a user defined above or already present in XO.
groups:
- name: operators
users: [ops@example.com]
Yes I am interested in any and all feedback. As of now it uses the JSON-RPC API and auth token. It will pull the current backup job / settings and create them in the new XO. If there is decent interested in this I will work on adding in the other features as well.
As my home lab I am not using all the backup features so can only test basic backup job creating and such. IF other can test the Mirror jobs, DR, or Continuous Replication.
Edit - It uses both Rest API and JSON -RPC API..
REST API (/rest/v0) — used wherever it supports the operation: remote (backup repository) create/read/update, VM listing, etc. Client in src/client/rest.ts.
JSON-RPC (websocket, via Vates' xo-lib) — used where REST is still read-only (backup jobs, schedules) or missing an operation entirely (e.g. remote deletion — REST only has "forget"). Client in src/client/jsonrpc.ts.
Authentication is a REST token (POST /rest/v0/users/me/authentication_tokens) that covers both transports (cli.ts:33).
The design intent is that REST is preferred, and JSON-RPC fills the gaps until the REST API reaches parity — at which point only the facade file needs to change.
Hi all,
I've been working on a project I'd like to share and get feedback on: xo-apply — configuration-as-code for Xen Orchestra.
The idea: XO keeps its entire configuration locked inside its own database. There's no way to write any of it down in a reviewable file, so rebuilding an XO means re-creating it all by hand in the UI. xo-apply treats XO as code: declare your XO in a YAML file, keep it in git, and reconcile any instance to match it — the same model Terraform uses for cloud accounts.
Backups are where we started — that whole surface is complete today (remotes, jobs, schedules, DR/CR, metadata, mirror, sequences)
What you can do with it:
Rebuild fast — reinstall XO, reconnect your pools, then xo-apply apply config.yaml and your remotes/jobs/schedules are back without clicking through the UI.
Detect drift — xo-apply diff tells you when someone changed a job in the UI and it no longer matches the file (exits non-zero on drift, so it drops straight into CI).
Review changes in git — every config change is a commit with an author and a diff.
Clone configs — apply one file to several XO instances (e.g. keep staging and production identical).
A quick taste of the diff output:
Remotes:
Plan: 2 to create, 1 to update, 0 untracked
There's also an interactive terminal UI — just run xo-apply with no arguments and it walks you through connecting, picking a config file, and running diff/apply/export. And a plain export command to write down what you already have on an existing XO, so you can adopt it without starting from scratch.
What's covered today (the complete backup surface):
Backup remotes (NFS, SMB, S3, local)
VM backup jobs + schedules (delta/full, smart-mode tag selection, explicit VM lists)
Disaster Recovery / Continuous Replication (SR targets)
Pool metadata + XO config backups
Mirror backups
Backup sequences (run schedules one after another)
On the roadmap — this is where the project grows beyond backups toward managing the rest of XO, and where I'd love input on priorities:
How it works under the hood: it talks to XO over the network — the REST API where writes are supported, and XO's JSON-RPC websocket via xo-lib (the same client xo-cli uses) for backup jobs/schedules, since the REST API doesn't expose those writes yet. No state file is kept: the running XO is always the source of truth. It runs anywhere Node.js ≥ 20 does — Linux, macOS, Windows — and works against any current XO from sources or XOA.
Install is a one-liner from GitHub (details + full docs in the README):
npm install -g https://github.com/acebmxer/xo-apply/archive/refs/heads/main.tar.gz
What I'm looking for: feedback from this community — is this something people want? And if anyone's willing to test what's there today against their own XO, I'd love to hear how it goes. diff and export are read-only and safe to try; apply shows you the full plan and asks for confirmation before it changes anything (and refuses to run unattended without --yes). One heads-up for testers rebuilding onto a fresh XO: connect your pool(s) and set any remote-secret env vars first — the README covers both.
It's an independent community project, AGPL-3.0 (the same license as XO itself), and not affiliated with or endorsed by Vates. It stands on Vates' own xo-lib and xo-remote-parser libraries.
Repo: https://github.com/acebmxer/xo-apply
Note: this project was built with Claude Code from the ground up.
Perfect. Created a script to run on the Synology device at startup it enables to the remotes, then at shutdown it disables the remotes.
Due to high electric coast and current heat I am trying to keep things powered off until needed. So I setup a power schedule for my Synology device used for backups. However when the device is powered off XO becomes unresponsive. If I set the remote to disabled xo is stays alert.
What would be the suggested way to automate the enable and disabling the remote when the device is offline? If this is something do able by xo but would need to be built, let me know and I will add it to the V6 request list.