@acebmxer Hello, no. Let me describe my setup. Esxi + vcenter + veeam. Vcenter wakes up veaam at 9am. Veeam sends wol packet to the synology nas at 9.15. At 9.30 the backup starts and ends more or less at 10.00 Due to inactivity synology stops the hard disks, then after some time it shut itself off. Veeam just see the repo as unavailable. At 12.00 veeam auto shuts down. Migrating to xcpng + xo, veeam is retired. At first i added the remote to XO, believing it's like veeam. If the backend goes down it simply shows unavailable. Then i discovered is not at all like that when i found an inaccessible XO. After some chatgpt-ing i found out this "bug". I thought that the script would be sufficient, but it's not. Waiting for ping to fail and then disable the remote (cron every 5mins) still ends up in ps -T -p 738 -o pid,tid,stat,wchan:32,comm PID TID STAT WCHAN COMMAND 738 738 Ssl do_epoll_wait MainThread 738 753 Ssl do_epoll_wait DelayedTaskSche 738 754 Ssl futex_wait_queue V8Worker 738 755 Ssl futex_wait_queue V8Worker 738 756 Ssl futex_wait_queue V8Worker 738 757 Ssl futex_wait_queue V8Worker 738 758 Ssl futex_wait_queue SignalInspector 738 863 Dsl rpc_wait_bit_killable libuv-worker 738 864 Dsl rpc_wait_bit_killable libuv-worker 738 865 Dsl rpc_wait_bit_killable libuv-worker 738 866 Dsl rpc_wait_bit_killable libuv-worker And it's even worse because not only XO becomes unavailable, but even if i disable manually (or via script) the remote as soon as the backup ends, with the synology still ON, the linux os does not unmount the nfs. Today the remote was disabled before the nas shutdown and correctly XO kept working. But if at this point i want to restart the XO vm i must do this before shutdown umount -l /run/xo-server/mounts/9fc232df-4d0c-4a15-93b5-c0585c1dd9c9 otherwise it stays like this for at least 15 minutes, then finally reboots [image: image.jpeg] The ideal behaviour would be that XO does all by itself, but since we can't have it all, the closest thing i have in mind is Script that sends WOL to the nas, when it pings than enables the remote Script that monitors the backup acrivity. when the last job is done, it must disable the remote and also umount the linux os nfs mount. Then the nas could shut down automatically without issue. Since i'm not a dev i need to take my time with chatgpt and see i what i can do. Having said that, my concern is not about my homelab, it's about production. In my environment i can and want to simulate all kinds of malfuncions. In the previous message i said i still need to try to shut down the main nfs SR for vms and see how the host behaves. But if i decide to deploy XEN at my company, i cant have 1 remote unavailability cause the whole XO (or XOA in that case) fall apart. Hope it helps!!

Thanks for the versions. If I read the changelogs right, sm-3.2.12-25.2 is one build after the 25.1 that @dthenot mentioned and still carries its QCOW2 patches, and the release note for blktap-3.55.5-12.3 says it fixed a tapdisk crash during backups (https://xcp-ng.org/forum/post/109336). So unless I missed one, you already have every fix I know about. I think this might be a different bug. I can't tell from here why tap-ctl close times out on those snapshot disks, so that one is for @Team-Storage. I'd guess the SMlog from both hosts around one failed run is the first thing they'll ask for.

Your reading of the code matches what I see on master, as far as I can tell. The object.enabled === true check in _getHosts() seems to have come in with PR #10209, released in XO 6.7.1 in August, to stop the balancer migrating VMs onto disabled hosts. If I follow the code right, a host that XO shuts down goes through disableHost first, so the power-on step in performance-plan.js no longer sees it. That looks like a side effect of the fix to me. The docs still say the performance plan powers halted hosts back on (https://docs.xen-orchestra.com/manage-your-infrastructure/load_balancing#plan-modes), so I think one of the two is wrong. I haven't reproduced it and there may be a reason to filter halted hosts that I can't see from the code, so @Team-XO-Backend is better placed to say if your powerState === 'Halted' exception is the right fix.

This is a question for @dinhngtu

error via reverse proxy. Uncaught Error: Body is not a JSON, original message is : <html> <title>500 Internal Server Error</title> <style>html{color-scheme:light dark;}</style> </head> <body> <center><h1>500 Internal Server Error</h1></center> <hr><center>Angie</center> </body> </html> pr index.js:2564 me index.js:4029 i index.js:4029 promise callback*me index.js:4029 i index.js:4029 promise callback*me index.js:4029 i index.js:4029 promise callback*me index.js:4029 i index.js:4029 be index.js:4029 be index.js:4029 dr index.js:2531 importDisks index.js:2575 Lodash 2 importDisks index.js:2574 e index.js:113 P index.js:22 i index.js:22 R index.js:22 R index.js:22 import index.js:121 t factory.js:307 o factory.js:331 e action-button.js:93 m action-button.js:10 i action-button.js:10 _execute action-button.js:10 _execute action-button.js:10 _execute action-button.js:10 v action-button.js:140 componentDidMount action-button.js:147 React 14 componentWillUnmount factory.js:125 f factory.js:31 _dispatch factory.js:158 t factory.js:322 promise callback*t factory.js:313 o factory.js:331 value index.js:206 promise callback*value index.js:174 React 5

@IgorGlock What do you mean by "okay"? Backing up the VM with Commvault or...?

@tjkreidl said: @john.c Note that a two-host pool running HA can run into the split-brain issue if one of the hosts fails; it's highly recommended to have at least three hosts in an HA pool (or look into HA-Lizard if running a pool with two hosts and HA). Right but also considering the in development Twinstor for when it goes stable and in production!

Thanks @poddingue, that's helpful. I'll keep the satellites on the management interface then. @Team-Storage could you please have a look at the LinstorSR attach failure, the RPU evacuating the controller host, and the xenopsd leftovers after the failed sync point 4? I have the RPU traces, SMlog, xensource.log and the linstor-controller error reports archived if needed.

@vmpr Popping it behind an instance of BunkerWeb (https://www.bunkerweb.io/) or similar, with Xen Orchestra on a new IP address would help. As BunkerWeb is what’s known as a Web Application Firewall. It will help to protect your instance of Xen Orchestra, especially if you change the IP address of the appliance, and only give that friend the one that goes through BunkerWeb. As always don’t expose Xen Orchestra to the Internet, but if it’s absolutely necessary always use something like BunkerWeb (with BunkerNET) and only provide the BunkerWeb based IP address! Also run a check on all of your hosts and may be best to roll them a fresh (in case there’s an Advanced Persistent Threat). As well as a fresh instance of Xen Orchestra, and maybe encourage your friend while also doing yourself to start using hardware security keys, preferably ones that support biometrics and PINs. Alternatively other secure enclaves with cryptographic credentials held there. Additionally opt into the new encrypted datastore for Xen Orchestra on the new fresh Xen Orchestra and XCP-ng. Also make sure it’s completely up to date! Also if the hosts support it ensure that Intel TME or similar is enabled on the fresh installs of XCP-ng. Finally rotate all passwords associated with that system, including your Vates account password, especially if it’s linked to the Xen Orchestra instance hit.

@jr-m4 I did not took time to test Podman yet. But we expect this to work. You'd have to expose a socket, like the docker daemon does. Probably systemctl enable --now podman.socket would be enough. More in this page https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/building_running_and_managing_containers/using-the-container-tools-api and this one https://docs.podman.io/en/latest/markdown/podman-system-service.1.html

@gudge25 check these https://docs.xcp-ng.org/vms/#enabling-viridian-extensions and https://lazyadmin.nl/it/september-2026-update-break-rds-how-to-fix/

Thanks @ataxyanetwork I was about to answer the same...! (but was too busy this morning )

Hi @poddingue ..yes, I have quite a few 'requests' I need to post in there...I just have yet to do so Thanks!

For the partitioning part, the docs say NVIDIA vGPU isn't supported on XCP-ng, since it depends on a proprietary XenServer component that can't be redistributed: https://docs.xcp-ng.org/compute#vgpu. So as far as I can tell, the "20, 40 or 80 GB per VM" split won't work through vGPU. Passing a whole GPU to one VM is the documented path, on the same page. I haven't seen anyone report on H100 or H200 cards here, though, so I can't tell you if that class of card brings its own surprises. There's also a blog post doing LLM inference that way on a much smaller Tesla T4: https://xcp-ng.org/blog/2025/07/28/your-own-gpu-powered-llms-with-xcp-ng/. There's an open design discussion upstream in XAPI about GPU partitioning, MIG included (https://github.com/xapi-project/xen-api/pull/7260), but that's the XenServer side, and I don't know if or when it would reach XCP-ng. As GPU partitioning matters for your project, it might be worth opening a request on https://feedback.vates.tech/ with what you described here (H200s, LLM inference, several tenants per card, fixed VRAM per VM). From what I've seen, a concrete use case like yours carries more weight there than a plain "please support vGPU", and other people needing the same thing can vote on it.

Hello all, Version 9.2.445 of the XCP-ng Windows Guest Tools has been released. Download the latest release here: https://github.com/xcp-ng/win-pv-drivers/releases This release brings important bug fixes to the Windows drivers and installer. Important note for users upgrading from 9.2.350: You must first install the XenTools-fix-9.2.351.msp hotfix to upgrade to 9.2.351 before installing 9.2.445. Otherwise, upgrades from 9.2.350 will be blocked. Reboot is not required after installing the hotfix. The installer downloads include a copy of XenClean and XenBootFix. It is not necessary to run XenClean when upgrading from previous versions of the XCP-ng Windows guest tools, unless prompted by the installer. Remember to carefully read XenClean docs before use. Note: Always download the latest XenClean/XenBootFix from https://github.com/xcp-ng/win-pv-drivers/releases . You can find them in the latest release ZIP/ISO archives. Before installing Thank you for using our Windows PV tools. Please carefully read the instructions below. It is not necessary to run XenClean when upgrading from previous versions of the XCP-ng Windows guest tools, unless prompted by the installer. Not compatible with the "Manage Citrix PV drivers via Windows Update" option. You must disable this option before installing. Changes since 9.2.385 NEW: Group Policy support (see the extras/admx directory in the source code tree). Supported policies include: Hide the tray icon Disable all features that allow controlling the VM from the host Improved: Better network performance. Improved: System reliability improvements on driver upgrade failure. On an upgrade failure, the installer will remove all Xen drivers to ensure that the system will keep operating, and to allow you to safely uninstall the existing driver package. Improved: Remove obsolete Windows Update devices and don't block installation because of them. Fixed: Fix further data corruption bugs during memory snapshots and backups. This driver version significantly narrows the corruption window. However, fully fixing all known corruption issues requires further work in other XCP-ng components. Please keep an eye on the XCP-ng blog for updates. Fixed: Fix loss of VBD connection after storage migration. Fixed: Improve reliability of the clipboard service. Fixed: Improve reliability of the VIF IP configuration feature. Fixed: Reliability fixes on VMs with many vCPUs (>64). Fixed: Many other driver reliability fixes. Help / Community support If you encounter installation/uninstallation errors, please try again with one of the following commands: For installing: msiexec.exe /i XenTools-x64.msi /l*vx install.log For uninstalling: msiexec.exe /x XenTools-x64.msi /l*vx uninstall.log Please include this log along with the file C:\Windows\INF\setupapi.dev.log in your bug report. These files will help us troubleshoot any installation issues. Additionally, please report any errors or BSOD you encounter during testing of this release. Your feedback is very appreciated.

@poddingue Yes, reads pretty much like the same error / root cause.

Thanks, that helps. Some VMs fail to the Synology but pass to S3, and some the other way round. So I think the backups on Ark are probably fine (they pass health checks, as you said at the start) and it has more to do with what each target already holds, but I could be wrong. I'd also flag the several delta jobs with different retentions all feeding Ark, I just don't know the mirror code well enough to say whether that matters. @Team-XO-Backend, posts 6 and 10 have the log and the pattern if someone gets to this one.