• Error mirroring full backups to backblaze b2

    Unsolved Backup
    28
    2
    0 Votes
    28 Posts
    3k Views
    P
    @pierrebrunet Hi! I'm on vacation now, so sorry for the delay. In the meantime I've been analyzing these issues and it seems these are somewhat expected and still happen with minpartsize set to 100M. My next step is to increase the AWS_MAX_ATTEMPTS from the default 3 attempts to 10. This might reduce the number of failed VMs and avoid the need to retry the full VM backup. From AI research, it seems rclone uses 10 attempts instead of the AWS SDK default. Here are some stats of my weekly full backup to B2: "00 Weekly Full Backblaze" ran xxxx → xxxxx — 34.1 hours total, moving ~2.86 TiB across 89 VMs. Final status: failure (more on that below, because it's misleading) -> this is related to XO reporting the backup as a failure even though the follow up retry succeded which is a known issue. Every one of the 7 first-attempt errors shows attempts: 3 in the SDK metadata — the default budget was exhausted in all cases -> 7 of 89 VMs failed on the initial attempt. Error mix: 5× Backblaze InternalError 500, 1× ECONNRESET, 1× EPIPE The 7 failed first attempts burned ~7.1 hours of cumulative transfer time Even though XO does not expose AWS_MAX_ATTEMPTS parameter I might be able to set it using an environment variable. I'll give a try when I'm back at the office. Tks, Pedro
  • CBT: the thread to centralize your feedback

    Pinned Backup
    456
    1 Votes
    456 Posts
    814k Views
    G
    Hello, Just a feedback. I am having trouble using CBT Backups with NBD. Host Details: XCP-NG Version : 8.3 latest (stable) Xen Orchestra: latest:stable (From Source) The first full backup always seems to work. The delta backup gives the error: 'HANDLE_INVALID' the writer IncrementalXapiWriter has failed the step writer.checkBaseVdis() with error HANDLE_INVALID(VBD, OpaqueRef:xxxxx). It won't be used anymore in this job execution. Start: 2026-08-09 14:01 End: 2026-08-09 14:01 Duration: a few seconds Error: HANDLE_INVALID(VBD, OpaqueRef:xxxxxx) This is a XenServer/XCP-ng error
  • 0 Votes
    3 Posts
    96 Views
    olivierlambertO
    No problem, this is always good to have feedback and cases showing an error and the explanation
  • Existing AD Users Cannot Login to XOCE but New Users Can

    Solved Xen Orchestra
    19
    0 Votes
    19 Posts
    4k Views
    K
    RESOLVED — root cause found, three years later. Leaving a full write-up for anyone who lands here from a search. Short version: this was never an XO bug, and it was never intermittent. The answer was sitting in the very first test-cli.js output I posted back in May 2023, and I misread it — as did everyone else in this thread, myself very much included. The line that mattered failed to bind as CN=Agbasi\, Kismet,...: 80090308: LdapErr: DSID-0C090434, comment: AcceptSecurityContext error, data 569, v4f7c We all pattern-matched AcceptSecurityContext error to "bad credentials" and moved on. But the meaning is entirely carried by the data field, which is the underlying Win32 status in hex: data 52e = 0x52E = 1326 = ERROR_LOGON_FAILURE — this is the "wrong password" one data 525 = 1317 = ERROR_NO_SUCH_USER data 532 = 1330 = password expired data 775 = 1909 = account locked out data 569 = 0x569 = 1385 = ERROR_LOGON_TYPE_NOT_GRANTED I was getting 569, not 52e. My password was correct all along. AD validated it, then refused the logon type. Why that happens xo-server-auth-ldap verifies a password the only way LDAP allows — it re-binds to the directory as the user. Against Active Directory, an LDAP simple bind to a DC is processed as a Type 3 (network) logon on that DC. So if an account is caught by "Deny access to this computer from the network" (SeDenyNetworkLogonRight) in the Default Domain Controllers Policy (or any other WINNING GPO, for that matter), it cannot complete an LDAP bind — no matter how correct the password is, and no matter which LDAP client is asking. My environment uses a tiered admin model. Non-domain-admin admin groups are explicitly denied network logon to the DCs. My admin account is in those groups. Hence 569, every single time, by design. Why it looked intermittent It wasn't. I sampled it either side of a config change. I could prove a bind had succeeded recently, because my LDAP-only XO account (no local password on the record at all) minted an API token on 28 July. Then on 31 July I restored RBAC settings on the Default Domain Controllers Policy that had drifted at some point — I found that during unrelated PKI work. GptTmpl.inf last-write confirms it. The token's last successful use is about eleven hours before that edit. Two deterministic states, one config change in the middle. That's the whole "intermittency." My 2023 "seven security groups" theory was wrong For the record, since it's still up there and someone will find it: I removed group memberships one at a time until auth worked, and concluded there was a membership count limit. There isn't. My own control test disproved it at the time — adding fifteen groups never reproduced the failure — and I should have taken that seriously instead of filing it under "weird." The variable was never the count. It was which group. One of the removals happened to drop the account out of a denied group. My other closing theory in this thread — special-character handling in the username or password — was also wrong. Getting 569 back proves AD parsed the escaped DN (CN=Agbasi\, Kismet), found the object, and got as far as evaluating the password. A mangled DN gives you 525 or a DN syntax error, not a logon-rights rejection. ldapts and passport were behaving correctly throughout. How to check this in 60 seconds Run the plugin test CLI and note the data value. Convert hex → decimal, look it up in Microsoft's System Error Codes list. On the DC, look for Security event 4625 with Sub Status 0xC000015B (STATUS_LOGON_TYPE_NOT_GRANTED). Fastest test of all — from a workstation, as the affected account: net use \\dc01\sysvol. If network logon to the DC is denied, this fails too, and you've confirmed it without touching XO at all. Check the policy directly: $p = "\\mydomain.net\SYSVOL\mydomain.net\Policies\{6AC1786C-016F-11D2-945F-00C04fB984F9}" + "\Machine\Microsoft\Windows NT\SecEdit\GptTmpl.inf" Select-String -Path $p -Pattern "SeDenyNetworkLogonRight|SeNetworkLogonRight" Resolve the SIDs and see whether your user is in any of the denied groups. Also worth checking your grant side: if Access this computer from the network doesn't list Authenticated Users directly, ordinary users are probably getting it transitively via Pre-Windows 2000 Compatible Access. Worth confirming before you assume a plain non-privileged account will work. What I am NOT doing Removing those groups from the deny right. It's doing exactly what I rebuilt it to do. Restoring an app login by handing admin groups network access to the DCs for SMB/RPC/LDAP is a bad trade, and I'd just be undoing my own remediation. Fix Interim: local XO accounts for the admins who need them. No AD objects created, nothing to unwind later, per-user attribution preserved in the audit log. Long term: federate XO through Keycloak (OIDC) instead of LDAP. Kerberos ticket issuance is a KDC service operation and is not gated by SeNetworkLogonRight — which is exactly why these accounts log into workstations all day while failing an LDAP bind. ️ Important if you go the Keycloak route: Keycloak's LDAP user federation validates passwords by doing an LDAP bind. Configure it that way and you'll hit data 569 inside Keycloak instead of inside XO and gain nothing. Password validation has to be delegated to Kerberos/GSSAPI. This will bite you on anything else you point at LDAP too — Bitwarden, NPM, TrueNAS, the lot. Worth solving once at the IdP. One request for Vates @olivierlambert @julien-f — you were right that it was environmental, and I owe you both thanks for the time you put in back in 2023. That said, there's a real (small) improvement available here. xo-server collapses every auth provider exception into a generic invalid credentials, and the plugin only emits the actual AD error at DEBUG. The DC told us precisely what was wrong on the very first attempt — it just never reached anywhere a user would look. Surfacing the LDAP result code and the AD data sub-code at INFO on failure, and in the plugin test output in the UI, would turn this class of problem from a multi-year hunt into a single-session diagnosis. Happy to open an issue on GitHub with the full reproduction if that's useful. Hope this saves someone else three years. If you found this thread by searching data 569, *ERROR_LOGON_TYPE_NOT_GRANTED**, or "LDAP invalid credentials but password is correct" — check your Deny access to this computer from the network user right first. That's almost certainly it.
  • 0 Votes
    5 Posts
    162 Views
    M
    Hard reboot sorted itself out
  • XOA Updater fails

    Unsolved Xen Orchestra
    4
    0 Votes
    4 Posts
    593 Views
    andibingA
    @john.c said: nd need to be cleared, before re-attempting the update. Checking the logs will help Just for completeness... it kept doing this after all the typical diagnosis steps. So never found a solution Definitely wasn't cache or disc space related. In the end I built the XOA from source on the same VM and that worked fine. Although subsequent to that I've moved XOA to the containerised version thus saving an extra VM!.
  • Facing some issue in copy Function of Xen Orchestra

    Xen Orchestra
    4
    0 Votes
    4 Posts
    148 Views
    AtaxyaNetworkA
    @irtaza9 i would look a the usual suspect, /var/log/SMlog /var/log/xensource.log see https://docs.xcp-ng.org/troubleshooting/
  • Test results for Dell Poweredge R770 with NVMe drives

    Hardware
    36
    7
    0 Votes
    36 Posts
    11k Views
    gduperreyG
    @yllar Hello, We are still validating the ISOs, but we are getting closer to releasing them. We hope to make them available soon.
  • Veeam for Xen Orchestra has been release today 13.1

    Backup
    25
    0 Votes
    25 Posts
    1k Views
    acebmxerA
    So I ended up going with the Windows B&R as i could net setup the windows mount server when setting up the repo for the backups. It would setup the linux one but not the windows one. This was with using the veeam console from a windows client. So i ended up with the Windows one. Now after a few backups i am see this warning / error.... 8/5/2026 5:34:50 PM Warning : Failed to use CBT: [Task e75c4247-2ee1-7b51-088c-b970ace60f3d (Async.VDI.list_changed_blocks) failed: . SR_BACKEND_FAILURE_460. . Failed to calculate changed blocks for given VDIs. [opterr=Source and target VDI are unrelated] First i thought it was because i maped the new job to older backups from Beta v1. So i purged all old backups and started fresh. The full backups were successful. Now on first delta 4 out of 7 vms have that warrning. This a Veeam issue or xcp-ng?
  • VDI_IO_ERROR during manual import of xoa

    Management
    4
    2
    0 Votes
    4 Posts
    138 Views
    olivierlambertO
    What kind of storage are you using? Is it a 4K block native storage? You should be able to see if you can create a virtual disk easily with xe vdi-create name-label=test virtual-size=1G sr-uuid=356475ac-…
  • Get a price quote on a plan

    XCP-ng
    2
    0 Votes
    2 Posts
    90 Views
    DanpD
    Hi, I found your original request. I will make sure someone from the Sales team responds ASAP. Regards, Dan
  • V2V Migration Completion?

    Solved Migrate to XCP-ng
    2
    2
    1 Votes
    2 Posts
    100 Views
    acebmxerA
    If i recall from my migration I belive thats all there is to it. I know the docs mention about doing a test migration but mine just migrated and it just worked as you stated after making the needed adjustments. Let other chime in but i think you are good to go. And you could delete the snapshot from the vm.
  • 2 Votes
    14 Posts
    3k Views
    olivierlambertO
    Very cool, thanks for the feedback @maximsachs !
  • Pool metadata Restore RESTORE_INCOMPATIBLE_VERSION

    Moved Unsolved Backup
    5
    0 Votes
    5 Posts
    229 Views
    F
    This might have worked, at that time i've allready created new VMs from the information inside the Backup xml file. It worked out allright but building VMs this way, is quite tidious. Maybe there could be a way to find the correct installer and patches, to get a new host in a state to restore a pool backup, would be good.
  • Rolling Pool Update and Affinity Host

    Management
    5
    0 Votes
    5 Posts
    647 Views
    olivierlambertO
    Hi! No, but check XO logs/output to see what's going on
  • 0 Votes
    6 Posts
    251 Views
    K
    Filed both XO related issues: Memory visibility at VM creation: https://github.com/vatesfr/xen-orchestra/issues/10225 domain_crash invisible in XO: https://github.com/vatesfr/xen-orchestra/issues/10226 Linking to this thread as promised earlier, thanks all. Hope this helps someone out.
  • 0 Votes
    3 Posts
    176 Views
    RN0R
    @poddingue, that is true, my first post here. I'll try to play with these BIOS settings again. and keep you posted. Cheers
  • ASUS NUC NUC14MNK-B LAN problems

    Hardware
    12
    0 Votes
    12 Posts
    811 Views
    yannY
    Drivers are actually in @Team-Hypervisor-Kernel scope nowadays
  • 2 Votes
    5 Posts
    277 Views
    A
    @poddingue Borrow away - "smoke alarm" is a better name for it than anything we had, so we might borrow it right back. Since you mentioned reading the matrix - here is the classification you would be reading, straight from the file: grep access: dadl/xen-orchestra.dadl | sort | uniq -c 49 access: admin 21 access: dangerous 122 access: read 75 access: write 122 of the 267 tools are plain read - that is the entire surface a review-capped agent gets. The other 145 exist in the same file, but for that agent they might as well not. The whole security taxonomy is greppable plaintext - which is rather the point of a declarative format. And if anything in the matrix looks wrong or missing, this thread is exactly the right place - real-world corrections are how it improves.
  • PCIe Passthrough of Radeon iGPU fails

    Unsolved Hardware
    13
    0 Votes
    13 Posts
    1k Views
    S
    @yannsionneau , may this link helps if you not anyway know it https://medium.com/@timemaster5/getting-amd-phoenix-igpu-passthrough-working-on-xen-xcp-ng-8856593f9c0d I will give it a try this weekend