Subcategories

  • VMs, hosts, pools, networks and all other usual management tasks.

    499 Topics
    4k Posts
    F
    @dinhngtu I will mention that the package was deployed via windows SCCM deployment tools so that may have also played a part in leaving the old product code behind. After running the above script we were able to deploy 9.2.445 via SCCM without issue and no old product codes were left behind.
  • ACLs, Self-service, Cloud-init, Load balancing...

    107 Topics
    877 Posts
    Bastien NolletB
    Hello @ludalm13, Thanks for your contribution and sorry for not giving an answer earlier. We initially planned to review your PR while doing some additional work on the audit plugin, but our planning evolved and that work was postponed. We'll review your PR in the following days.
  • All XO backup features: full and incremental, replication, mirrors...

    550 Topics
    6k Posts
    acebmxerA
    Now that there is more progress in v6 specifically in the backup section. Not sure if this is a bug or as intended, leaning towards a bug. This is my existing backup job that works just fine in v5 with sequences task setup. The below recommendation would that create two separate backup chains, one of deltas, and one of full backups? Also it will not let me to manually run the backup job as it states the job has more then one schedule. Node - v26.13.1 Commit - 3cd44 [image: Screenshot_20261117_194957.png] [image: Screenshot_20261117_195026.png] [image: Screenshot_20261117_195450.png]
  • Everything related to Xen Orchestra's REST API

    89 Topics
    656 Posts
    P
    Hi, This was fixed in Latest. But if you need to stay on Stable, you can open a ticket to have a patch.
  • Terraform, Packer or any tool to do IaC

    54 Topics
    507 Posts
    D
    For anyone else who steps through this, I've also installed the latest xen-guest-agent with the below process. Repeat for each host in the environment, 1 at a time from your Helper VM. ### ssh into your helper ### oc debug node/fedora-cmp-1/2/3 chroot /host cd /var/tmp curl -LO https://gitlab.com/xen-project/xen-guest-agent/-/jobs/6041686376/artifacts/raw/RPMS/x86_64/xen-guest-agent-0.4.0-0.fc37.x86_64.rpm sudo rpm-ostree install /var/tmp/xen-guest-agent-0.4.0-0.fc37.x86_64.rpm mkdir /var/tmp/xga && cd /var/tmp/xga rpm2cpio ../xen-guest-agent-0.4.0-0.fc37.x86_64.rpm | cpio -idmv ### From your helper... DRAIN YOUR WORKERS ### oc adm drain fedora-cmp-1/2/3 --ignore-daemonsets --delete-emptydir-data ### log back into fedora-cmp-1/2/3... ### sudo systemctl reboot ### from helper ### oc adm uncordon fedora-cmp-1/2/3 ### login to fedora-cmp-1/2/3 ### chroot /host sudo systemctl enable --now xen-guest-agent And then from XO this is what your VMs will look like. [image: image.jpeg]
  • 0 Votes
    29 Posts
    8k Views
    A
    Apologies for the long silence on my end — real life got in the way and I couldn't dig into this for a while. Finally found the time now to go through it again properly, helped a lot by the details others have already posted here in the meantime. I don't think this adds a new root cause, but it does corroborate what @kagbasi-wgsdac described — same "mass collapse onto a single anchor UUID" pattern, and in my case too that anchor is a real, still-in-use base disk, not a dangling/non-existent OpaqueRef (referencing vatesfr/xen-orchestra#9578). Environment: 2-host pool, shared NFS SR, XO recently updated to the latest version (didn't change anything regarding this issue, as expected since it's XAPI-side). Ran a read-only xe vdi-list params=uuid,name-label,is-a-snapshot,snapshot-of,snapshot-time directly against the pool master. Result: 47 of 196 VDIs match the pattern (is-a-snapshot: false but snapshot-of populated). All 47 point to the exact same single anchor UUID — which resolves to one specific, actively-used base VDI, not a missing reference. snapshot-time on most affected VDIs is the epoch default (1970-01-01T00:00:00Z), but a handful show plausible real dates (e.g. late 2025 / early 2026) — suggesting those entries originally had a legitimate snapshot-of relationship that got overwritten by whatever corrupted the metadata. Currently-running VMs affected (100% of each VM's disks affected in every case): VM OS Disks Sizes VM-1 Windows Server 2016 1 50 GB VM-2 Windows Server 2016 3 50 / 200 / 450 GB VM-3 Windows Server 2019 3 100 / 100 / 100 GB VM-4 Windows 11 Pro 2 50 / 100 GB VM-5 (decommissioned) Windows Server 2016 2 100 / 50 GB VM-6 Ubuntu 24.04 2 10 / 10 GB VM-7 Ubuntu 24.04 2 10 / 10 GB VM-8 Windows Server 2025 3 25 / 25 / 64 GB VM-9 Ubuntu 24.04 2 10 / 10 GB On top of that, ~28 more affected VDIs are orphaned/unattached objects ("base copy" leftovers, old ISO references) — same pattern, no VBD attached. Not touching any of this (no snapshot-fixer.py, no manual vdi-param-set) given the risk of severing legitimate snapshot relationships that's already been flagged here. Happy to provide a full anonymized xe vdi-list dump if that's useful for tracking down the root cause. Big thanks to everyone who kept digging into this and shared their findings here — especially @kagbasi-wgsdac for the detailed write-up that pointed me in the right direction, and of course @poddingue and the whole Vates team for staying on top of this and keeping us updated despite no clear timeline yet. Really appreciate the effort that goes into this, especially for something as tricky as a metadata corruption bug across production pools. Kind Regards and thx again Alex
  • VDI export to VMDK results in a corrupted disk

    Solved
    15
    0 Votes
    15 Posts
    1k Views
    A
    @Emmanuel-V In my opinion, it would make more sense if a standalone disk exported in VMDK format were exported directly as monolithicSparse, so that it would not need to be converted. This makes more sense to me because when only the disk is exported, rather than the entire VM (OVA), it can be assumed that the disk will be attached directly to some VM.
  • XO NFS option sec=krb5p encrypted transport

    3
    0 Votes
    3 Posts
    399 Views
    BytevenidosB
    I do have active directory setup in my lab environment. I just need to set aside some time to try it out between two domain joined hosts. Thanks for trying that! That's some good debugging.
  • XCP Pulse collects XCP-ng and Xen Orchestra logs

    4
    3
    3 Votes
    4 Posts
    578 Views
    acebmxerA
    v0.8.0–v0.9.1 — multi-user accounts, self-update, built-in HTTPS, and date ranges. Three releases since the last update, so bundling them here. Date ranges. Collections, extractions, findings runs and the support package can now be scoped to a window instead of always covering the whole bundle — last 24h/7d/30d, since last reboot, or a custom range. XO's own routes still have no date filter, so the first download is unchanged size — the range only narrows what gets kept and reported afterwards. Redact on demand. A collection no longer has to redact immediately with whatever rules happen to be on at that moment. There's now a checkbox to store the raw bundle only, and a "Redact now" button on the card afterwards using whichever rules are switched on when you press it. Multiple user accounts, roles, and an activity log. This was on the "what is coming" list in the first post and it's in now. Any number of accounts, three roles — admin, operator, viewer (read-only, but can still download what's already stored) — and an Activity page logging logins, settings changes, and job runs. Optional 2FA. Per-user TOTP, off by default, each account turns it on for itself. QR code setup, backup codes. Self-update from the UI, opt-in and off by default. Checking needs nothing extra; applying an update needs the Docker socket mounted in explicitly, since that's effectively host root, so it's a separate deliberate step — uncomment the socket mount and group_add block in docker-compose.yml, and it needs its own .env file (just DOCKER_GID=..., next to docker-compose.yml, not the same file as xcp-pulse.env) or docker compose up -d fails with unable to find group. .env.example has the one-liner to generate it. Built-in HTTPS, no reverse proxy required. Bundles nginx into the image to terminate TLS — self-signed cert on first start, or upload your own from Settings. A reverse proxy still works fine too. A user manual in the app itself, under /help — no need to leave XCP Pulse or check out the repo to read how something works. Fixed The account-requirements note from the first two posts was wrong in a way I only found by testing it properly: export:logs — the privilege log downloads actually need — isn't in any built-in XO role, but it can be granted through a custom role. "Test connection" was reading a catalogue that can't tell you that, and told every restricted account it needed full admin regardless. It now actually probes the download endpoint, and the docs walk through creating the custom role via three REST API calls (no UI for it yet on either XO version). A real concurrency bug: the shared SQLite connection wasn't locked across fetches, only execute, so two requests landing close together could crash a page reading jobs — reproduced it under the test suite's own concurrency test. Base-OS CVEs patched in the image build (perl-base, libc6, others); CI now runs a Trivy scan on every push. Still not fixed: the truncated-download-behind-a-reverse-proxy issue from the last post. Haven't found the actual cause yet. Also looking for anyone who can test against a remote proxy in a lab setup. Also open to any other suggestions, features, improvements, UI changes, etc... If any chance someone on Vates would test on their own time. Things like the support bundle and or the logs themselves are not being manipulated in any unwanted ways (for Vates or the project) https://github.com/acebmxer/xcp_pulse
  • search for snapshots sorted by creation date

    Unsolved
    14
    0 Votes
    14 Posts
    4k Views
    fred-stoF
    @Danp said: I haven't seen a way to do this from within XO. However, you can gather the details using xe on the command line -- for i in `xe snapshot-list | grep uuid | awk '{print$5}'`; do export j=$(xe snapshot-param-get uuid=$i param-name=children); k=$(xe snapshot-param-get uuid=$i param-name=snapshot-time); echo "VM UUID: $j - Snapshot UUID: $i - Creation Time: $k"; done This doesn't sort the results, so you will need to do that in Excel or your favorite text editor. @Danp adding | sort -t: -k4 to your command will sort it by date. -t: set fields separator to : -k4 sort by 4th field
  • RPU issue

    3
    1
    0 Votes
    3 Posts
    521 Views
    M
    @poddingue You're right. The RPU task shown in the list was a previous one. Even that one finished fine. So both RPU tasks didn't show at all today! My bad. The evacuation and remirroring tasks were all shown. Latest XOA 6.7.1
  • Feature request: Change bond mode in XO

    2
    0 Votes
    2 Posts
    350 Views
    poddingueP
    I measured this on a two-NIC 8.3 host. As far as I understand, there's no way to do it from XO: the call that edits a network doesn't take a bond mode, every bondMode in the codebase sits on a create path, and the REST API on a running XOA offers create_bonded_network plus get and delete on a network, with nothing that edits one. The CLI route looks cheap though. xe bond-set-mode on a live bond cost zero dropped packets in both directions I tried, pinging every 200ms across the change, and the command returned in about 1.4 seconds. Two caveats, and the second is about your case specifically: creating and destroying the bond did interrupt the host (roughly 8 and 16 seconds), and I only went active-backup to balance-slb and back, never touching lacp, because my switch port isn't configured as a LAG and I'd have dropped the box. So I can't tell you your lacp to active-backup move is free, only that mode changes in general didn't cost me anything. Worth putting on https://feedback.vates.tech either way so the votes have somewhere to gather.
  • ACL V2, we need your feedbacks!

    2
    4 Votes
    2 Posts
    1k Views
    poddingueP
    I'm late to this, but I've been building lately a JetBrains plugin against the REST API and ACL v2 turned out to decide its whole design, so here's some feedback. Everything below is just one appliance, one pool (my small homelab), on a plan 4 trial, with an admin control call taken in the same breath as every scoped one. What I experienced is that selectors narrow reads (tags: and id: both, 1 VM against an admin control of 11), they're re-evaluated per request rather than fixed when the privilege is created, deny composes the way your Carol example says (allow-all plus deny on a tag gave 9, which is 11 minus the 2 tagged), and they scope the power verbs too, not just reads. The event stream is, to me, scoped as well, which was the thing I most wanted to check, because a scoped read next to an unscoped feed would have been a nasty trap. It's not. With two streams open at once, a change to an out-of-scope VM produced an update on the admin stream and nothing at all on the scoped one, so it's genuinely filtered rather than merely quiet. And the bit I'd underline: the same single change is update to the admin and add or remove to the scoped user. The verb is computed per subscriber, not per object. Across that run the admin only ever saw update, and I originally wrote that an admin never sees add or remove at all. That was wrong and I've since measured it: an admin does get add and remove when an object is genuinely created or deleted. So the rule is that a scope change reads as update and an existence change as add or remove, which supports your design better than what I first wrote did. "From the user's perspective, not XOA's" turns out to be literal rather than a figure of speech, and that's a nicer piece of design than the sentence let me imagine. Now the three things that I didn't see in the post above. vm-snapshot is a separate privilege resource and vm doesn't imply it. Maybe that's obvious, but it was not obvious to me. With all six VM privileges granted, GET /vm-snapshots came back empty while admin saw 6, including a snapshot the scoped user had just taken. Anything with a restore or revert screen gets an empty list and no error. Inherited snapshot tags look like a snapshot-time copy rather than a link. Tag a VM and it's in scope immediately, but its existing snapshots keep tags: [] and stay invisible. So someone onboarded into a tag scope after their snapshots exist sees the VM and not its history. Subscribing delivers no initial dump. Both streams sat on init and keepalives until something changed, so it's a delta feed and a client has to fetch the collection over REST and maintain it from events. Worth a line, since the natural assumption (at least to me, don't make that the rule for everyone) is the other one. While I'm here: two smaller ones. Privilege action names aren't REST action names: shutdown:clean grants clean_shutdown, revert-snapshot grants revert_snapshot, and a mistyped action quietly gives you a privilege that grants nothing. And the event: init frame's field is id, not connectionId (why did I think it was connectionId, no idea, I thought it was "natural"), which cost me a while of thinking the stream was dead when I was posting to /events//subscriptions. Yes, I know, I should have read the documentation instead of experimenting in the dark, sending made-up field names in the wild. One last thing: selector is optional, so a privilege created without one reads back as {id, resource, action, effect, roleId} with no hint the field exists. I granted allow read on vm, saw all 11 VMs, read the object back, and (falsely) concluded the REST API had no object dimension at all. It's all in the previous post and it's in the swagger, and of course, in the official documentation. Once again, I'm an innocent victim because I didn't RTFM. I just never saw a privilege that had one. If a privilege echoed selector: null, or if the first example anyone met were a scoped one, I don't think I would have spent much time on that. Once again, my bad, didn't RTFM. This was a small, targeted test, at best. I didn't have the intent to test what was brought up in this very thread, I just happened to tinkle with the REST API and ACL V2 for my PoC, so lots of things got untested. Please, don't take it from me as settled: only tags: and id: selector forms, nothing on a second pool or a real multi-user deployment, and I have not checked what happens to a live subscription when the privilege itself changes rather than the VM's tags. If you read me until there, you're brave, or have too much time on your hands.
  • After Update XO wont start

    Unsolved
    6
    1
    1 Votes
    6 Posts
    1k Views
    D
    Hey, that sucks after catching up on 23 commits. Looks like something broke in the log module during the update. I’d try a clean reinstall of the dependencies first, or drop back to an older Node version for a bit since Node 24 can be fussy with these packages. When you need a quick mental break from troubleshooting npm issues, checking out TonyBet is a great way to enjoy some online gaming and sports betting. Hope getting those modules reinstalled gets your build running smoothly again soon!
  • Update Templates

    14
    1 Votes
    14 Posts
    4k Views
    D
    Hi bikemuch, For unregistered XOA the template auto-update is limited. Easiest way is via CLI: download the latest Debian 13 and CentOS Stream 10 ISOs, then use xe vm-import or create new templates from them. You can also check the XOA “Templates” section and force a refresh if available. Works fine for me this way.
  • Rolling pool update failed to migrate VMs back

    Unsolved
    17
    0 Votes
    17 Posts
    5k Views
    olivierlambertO
    @neal https://github.com/vatesfr/xen-orchestra/issues/10260
  • v6 UI VDI's not being shown all the time.

    Solved
    6
    3
    0 Votes
    6 Posts
    893 Views
    poddingueP
    @acebmxer No problem, that's excellent news, thanks for the feedback!
  • Existing AD Users Cannot Login to XOCE but New Users Can

    Solved
    19
    0 Votes
    19 Posts
    5k Views
    K
    RESOLVED — root cause found, three years later. Leaving a full write-up for anyone who lands here from a search. Short version: this was never an XO bug, and it was never intermittent. The answer was sitting in the very first test-cli.js output I posted back in May 2023, and I misread it — as did everyone else in this thread, myself very much included. The line that mattered failed to bind as CN=Agbasi\, Kismet,...: 80090308: LdapErr: DSID-0C090434, comment: AcceptSecurityContext error, data 569, v4f7c We all pattern-matched AcceptSecurityContext error to "bad credentials" and moved on. But the meaning is entirely carried by the data field, which is the underlying Win32 status in hex: data 52e = 0x52E = 1326 = ERROR_LOGON_FAILURE — this is the "wrong password" one data 525 = 1317 = ERROR_NO_SUCH_USER data 532 = 1330 = password expired data 775 = 1909 = account locked out data 569 = 0x569 = 1385 = ERROR_LOGON_TYPE_NOT_GRANTED I was getting 569, not 52e. My password was correct all along. AD validated it, then refused the logon type. Why that happens xo-server-auth-ldap verifies a password the only way LDAP allows — it re-binds to the directory as the user. Against Active Directory, an LDAP simple bind to a DC is processed as a Type 3 (network) logon on that DC. So if an account is caught by "Deny access to this computer from the network" (SeDenyNetworkLogonRight) in the Default Domain Controllers Policy (or any other WINNING GPO, for that matter), it cannot complete an LDAP bind — no matter how correct the password is, and no matter which LDAP client is asking. My environment uses a tiered admin model. Non-domain-admin admin groups are explicitly denied network logon to the DCs. My admin account is in those groups. Hence 569, every single time, by design. Why it looked intermittent It wasn't. I sampled it either side of a config change. I could prove a bind had succeeded recently, because my LDAP-only XO account (no local password on the record at all) minted an API token on 28 July. Then on 31 July I restored RBAC settings on the Default Domain Controllers Policy that had drifted at some point — I found that during unrelated PKI work. GptTmpl.inf last-write confirms it. The token's last successful use is about eleven hours before that edit. Two deterministic states, one config change in the middle. That's the whole "intermittency." My 2023 "seven security groups" theory was wrong For the record, since it's still up there and someone will find it: I removed group memberships one at a time until auth worked, and concluded there was a membership count limit. There isn't. My own control test disproved it at the time — adding fifteen groups never reproduced the failure — and I should have taken that seriously instead of filing it under "weird." The variable was never the count. It was which group. One of the removals happened to drop the account out of a denied group. My other closing theory in this thread — special-character handling in the username or password — was also wrong. Getting 569 back proves AD parsed the escaped DN (CN=Agbasi\, Kismet), found the object, and got as far as evaluating the password. A mangled DN gives you 525 or a DN syntax error, not a logon-rights rejection. ldapts and passport were behaving correctly throughout. How to check this in 60 seconds Run the plugin test CLI and note the data value. Convert hex → decimal, look it up in Microsoft's System Error Codes list. On the DC, look for Security event 4625 with Sub Status 0xC000015B (STATUS_LOGON_TYPE_NOT_GRANTED). Fastest test of all — from a workstation, as the affected account: net use \\dc01\sysvol. If network logon to the DC is denied, this fails too, and you've confirmed it without touching XO at all. Check the policy directly: $p = "\\mydomain.net\SYSVOL\mydomain.net\Policies\{6AC1786C-016F-11D2-945F-00C04fB984F9}" + "\Machine\Microsoft\Windows NT\SecEdit\GptTmpl.inf" Select-String -Path $p -Pattern "SeDenyNetworkLogonRight|SeNetworkLogonRight" Resolve the SIDs and see whether your user is in any of the denied groups. Also worth checking your grant side: if Access this computer from the network doesn't list Authenticated Users directly, ordinary users are probably getting it transitively via Pre-Windows 2000 Compatible Access. Worth confirming before you assume a plain non-privileged account will work. What I am NOT doing Removing those groups from the deny right. It's doing exactly what I rebuilt it to do. Restoring an app login by handing admin groups network access to the DCs for SMB/RPC/LDAP is a bad trade, and I'd just be undoing my own remediation. Fix Interim: local XO accounts for the admins who need them. No AD objects created, nothing to unwind later, per-user attribution preserved in the audit log. Long term: federate XO through Keycloak (OIDC) instead of LDAP. Kerberos ticket issuance is a KDC service operation and is not gated by SeNetworkLogonRight — which is exactly why these accounts log into workstations all day while failing an LDAP bind. ️ Important if you go the Keycloak route: Keycloak's LDAP user federation validates passwords by doing an LDAP bind. Configure it that way and you'll hit data 569 inside Keycloak instead of inside XO and gain nothing. Password validation has to be delegated to Kerberos/GSSAPI. This will bite you on anything else you point at LDAP too — Bitwarden, NPM, TrueNAS, the lot. Worth solving once at the IdP. One request for Vates @olivierlambert @julien-f — you were right that it was environmental, and I owe you both thanks for the time you put in back in 2023. That said, there's a real (small) improvement available here. xo-server collapses every auth provider exception into a generic invalid credentials, and the plugin only emits the actual AD error at DEBUG. The DC told us precisely what was wrong on the very first attempt — it just never reached anywhere a user would look. Surfacing the LDAP result code and the AD data sub-code at INFO on failure, and in the plugin test output in the UI, would turn this class of problem from a multi-year hunt into a single-session diagnosis. Happy to open an issue on GitHub with the full reproduction if that's useful. Hope this saves someone else three years. If you found this thread by searching data 569, *ERROR_LOGON_TYPE_NOT_GRANTED**, or "LDAP invalid credentials but password is correct" — check your Deny access to this computer from the network user right first. That's almost certainly it.
  • XOA Updater fails

    Unsolved
    4
    0 Votes
    4 Posts
    860 Views
    andibingA
    @john.c said: nd need to be cleared, before re-attempting the update. Checking the logs will help Just for completeness... it kept doing this after all the typical diagnosis steps. So never found a solution Definitely wasn't cache or disc space related. In the end I built the XOA from source on the same VM and that worked fine. Although subsequent to that I've moved XOA to the containerised version thus saving an extra VM!.
  • Facing some issue in copy Function of Xen Orchestra

    4
    0 Votes
    4 Posts
    499 Views
    AtaxyaNetworkA
    @irtaza9 i would look a the usual suspect, /var/log/SMlog /var/log/xensource.log see https://docs.xcp-ng.org/troubleshooting/
  • Deploy VM via cloud-init config

    Solved
    9
    1
    0 Votes
    9 Posts
    1k Views
    MathieuRAM
    @acebmxer Thank you for your quick feedback.
  • Unable to fetch latest master commit.

    Solved
    17
    1
    1 Votes
    17 Posts
    2k Views
    TS79T
    @acebmxer all good and thank you again for sharing your discovery on the forums
  • Feature request - VM folders

    feature request
    21
    3 Votes
    21 Posts
    7k Views
    olivierlambertO
    Perfect, thanks for your feedback!
  • Xenorchestra V6 issue

    6
    1
    0 Votes
    6 Posts
    1k Views
    pdoniasP
    @acebmxer I can confirm that, we'll fix it, thanks
  • VDI_IO_ERROR(Device I/O errors) Immediate HELP needed Please.

    3
    0 Votes
    3 Posts
    671 Views
    poddingueP
    You may have worked it out yourself already. A consistency check reporting inconsistent parity on Virtual Disk 1, plus Buffer I/O error on several dm- devices, is the storage layer underneath XCP-ng telling you something is wrong down there. The VDI_IO_ERROR is mostly XCP-ng saying it could not read the disk, not the cause itself. I would be careful about anything that writes to that array until someone who knows hardware RAID recovery better than I do has looked at it. I honestly don't know whether a rebuild helps or makes things worse from this state, and I'd rather say that than guess with your data. Might be worth a mention to @Team-Storage.