• 9 Votes
    52 Posts
    18k Views
    bvitnikB
    @benapetr Do you have some way or script to quickly iterate build->test workflow locally? That is without building a package, then installing on a system then testing. I'm running Debian based system.
  • New Tools 9.2.445 update failure

    Management
    2
    1
    0 Votes
    2 Posts
    54 Views
    F
    Looks like we ended up with a corrupt duplicate install in the MSI installer database. After some googling and learning how the installer determines the currently installed version from the tools github page i was able to come up with a command to check what version windows thought was installed, i noticed on working installs no error was output however on broken servers the command would end with an error. PS C:\xcpng-winpv-9.2.445.0-Release-x64\xcpng-winpv-9.2.445.0-Release-x64\package> $Installer.ProductsEx("", "", 7) | ForEach-Object { [PSCustomObject]@{ ProductName = $_.InstallProperty("ProductName") ProductVersion = $_.InstallProperty("VersionString") ProductCode = $_.ProductCode() } } | Where-Object { $_.ProductName -like "*XCP-ng*" } ProductName ProductVersion ProductCode ----------- -------------- ----------- XCP-ng Windows Guest Tools 9.2.385.0 {A9CDEFA1-AB19-426F-8065-64331E634A62} InstallProperty At line:2 char:5 + [PSCustomObject]@{ + ~~~~~~~~~~~~~~~~~~ + CategoryInfo : OperationStopped: (:) [], COMException + FullyQualifiedErrorId : System.Runtime.InteropServices.COMException I fed this output to Calude for a bit help and to shed some light on the issue and its solution, it first had me check for duplicate install entries: [image: image.jpeg] After confirming the presence of a unreadable entry it was able to create a powershell script to remove the bad entry. From the LLMs output: This script computes the packed GUID itself (no hand-conversion errors), exports a backup of each key, and does a dry run by default so you can see what it would delete. Run it from an elevated, 64-bit PowerShell. How to use it: Run it as is. It prints the packed GUID (confirm it matches 970CA0A0E00DBC0488B2F1DE377268AA) and lists which keys exist. If the list looks right, set $DryRun = $false and run again. It tries msiexec /x first and only falls back to registry deletion if that fails. Note that on the real run it attempts the msiexec step before any registry work, and the registry section runs even if msiexec fails. Re-run your listing script. Only the 9.2.385.0 row should remain. Caveats: This leaves a few residual references behind, such as entries in ...\Installer\UserData\S-1-5-18\Components and ...\Installer\UpgradeCodes. Windows Installer ignores these once the product keys are gone, so they’re harmless for your purposes. If the bad entry turns out to be a per-user install rather than machine-wide, the keys will live under a user SID instead of S-1-5-18. The “Not present” lines would tell you that. In that case, tell me and I’ll adjust the paths. The backup .reg files let you restore anything with a double-click if something goes wrong. fix_xcptools.txt I am by no means anywhere near an expert in powershell as i spend most of my time on Linux, this all AI generated but it was able to remove the entry and get the installer moving along on VMs where it would no proceed prior.
  • XCP-ng Windows PV tools announcements

    Moved News
    120
    0 Votes
    120 Posts
    66k Views
    F
    Running into an installation issue with this new version. I created a separate thread here about the issue: https://xcp-ng.org/forum/topic/12580/new-tools-9.2.445-update-failure
  • 1 Votes
    4 Posts
    195 Views
    Bastien NolletB
    Here's the fix, which should be merged in the following days/weeks, and should be in the next XO release: https://github.com/vatesfr/xen-orchestra/pull/10809
  • Backup failures after VHD to QCOW2 conversion

    Unsolved Backup
    7
    0 Votes
    7 Posts
    385 Views
    S
    Hi everyone Here are the logs (anonymized) for one complete backup run. Thanks for taking a look!
  • New 9.2.445 XCP Windows tools Roll Back

    Management
    3
    0 Votes
    3 Posts
    185 Views
    B
    Looks like the tools installation had an issue with Windows Update packages, the windows event log started throwing error relating to the cryptsvc service and C:\Windows\System32\catroot2 at the time I ran XenClean before trying to install 9.2445. Rebuilding Catroot resolved the issue and tools installed fine. net stop cryptsvc ren C:\Windows\System32\catroot2 catroot2.old net start cryptsvc Reboot Appears to be isolated to this single VM
  • 0 Votes
    12 Posts
    6k Views
    olivierlambertO
    Hi @odmsys It's on the roadmap and it won't wait for SMAPIv3. I truly hope we can bring something in a matter of few months, at least to have it to test
  • Native Ceph RBD SM driver for XCP-ng

    Development
    43
    3 Votes
    43 Posts
    12k Views
    A
    @dicode-nl Thank you for the code. I'm in testing now with ceph 20.2.5 and so far I haven't had any issues.
  • 0 Votes
    4 Posts
    232 Views
    R
    Hi, as promised, here are the log excerpts from the incident, in case they help the @Team-Storage folks (or anyone hitting the same thing). All times are CET (host time). VM dd25471a-fbd3-2b0c-df4c-b8182546f6a8, SR 1a97f391-…, migration task R:4fa24c2fefb3 (beta → gamma). 1. The failed attach (SMlog on gamma) The SR lock is acquired at 23:00:04 and the exception comes 6s later: Nov 10 23:00:04 gamma SM: [11963] lock: acquired /var/lock/sm/1a97f391-c733-8520-62c3-2f3847235a36/sr Nov 10 23:00:10 gamma SM: [11963] Raising exception [47, The SR is not available [opterr=Error: ERRO:NoneType]] Nov 10 23:00:10 gamma SM: [11963] ***** generic exception: vdi_attach: EXCEPTION <class 'xs_errors.SROSError'>, The SR is not available [opterr=Error: ERRO:NoneType] File "/opt/xensource/sm/SRCommand.py", line 157, in _run_locked target = sr.vdi(self.vdi_uuid) File "/opt/xensource/sm/LinstorSR", line 549, in wrap return load(self, *args, **kwargs) File "/opt/xensource/sm/LinstorSR", line 479, in load raise xs_errors.XenError('SRUnavailable', opterr=str(e)) One second later the same SR answers normally, which is why I think the failure was transient: Nov 10 23:00:11 gamma SM: [12120] Failed to join node(s): {'gamma', 'alpha'} Nov 10 23:00:11 gamma SM: [12120] Synchronize metadata... Nov 10 23:00:11 gamma SM: [12120] VDI c90eeb92-... loaded! (path=/dev/drbd/by-res/xcp-persistent-ha-statefile/0, hidden=0) 2. LINSTOR side (controller on beta) Three error reports between 22:59 and 23:00, all identical: ERROR REPORT 6AF3924C-00000-000001 (also -000002 and -000003) Version: 1.33.1 Node: beta Class name: com.linbit.linstor.netcom.PeerClosingConnectionException Generated at: Method 'connectionClosing', Source file 'TcpConnectorPeer.java', Line #627 *__checkpoint ⇢ Fetch thin capacity info 3. How it surfaces in xapi gamma xapi: [error|VDI.attach2] sm_exec failed with exception Storage_error ([S(Backend_error);[S(SR_BACKEND_FAILURE_47);[S();S(The SR is not available [opterr=Error: ERRO:NoneType]);S()]]]) gamma xapi: [error|Async.VM.pool_migrate R:4fa24c2fefb3|Storage_smapiv1_wrapper] dp:vbd/1/xvda sr:1a97f391-... vdi:60342e65-... op:attach(RW) error:Storage_error (...SR_BACKEND_FAILURE_47...) 4. The cleanup on the source also fails (beta) 23:00:10 beta xenopsd-xc: [error|71|Async.VM.pool_migrate R:4fa24c2fefb3|xenops_server] VM.migrate: Failed during Synchronisation point 4. msg: Xenops_interface.Xenopsd_error([S(Storage_backend_error);[S(SR_BACKEND_FAILURE_47);...]]) 23:00:10 beta xenopsd-xc: [error|71|...|task_server] Task 101 failed; Xenopsd_error([S(Does_not_exist);[S(VM);S(dd25471a-.../config)]]) 23:00:10 beta xenopsd-xc: [error|71|...|xenops_server] Triggering cleanup actions failed: Xenopsd_error([S(Does_not_exist);[S(VM);S(dd25471a-.../config)]]) So after the migration error, the cleanup fails too because the VM config is no longer in xenopsd on the source. That matches what I saw afterwards: XAPI says running on beta, but xenopsd doesn't know the VM. 5. What is left on the destination (gamma) The temporary extra/ entry is created at 22:58:09 and renamed to the final uuid at 23:00:03, i.e. before the attach fails at 23:00:10. Nothing removes it afterwards: 22:58:09 gamma xenopsd-xc: [debug|Async.VM.pool_migrate R:4fa24c2fefb3|xenops_utils] TypedTable: Writing extra/dd25471a-fbd3-2b0c-df4c-000000000001 23:00:03 gamma xenopsd-xc: [debug|13|Async.VM.pool_migrate R:4fa24c2fefb3|xenops_utils] TypedTable: Renaming extra/dd25471a-fbd3-2b0c-df4c-000000000001 -> extra/dd25471a-fbd3-2b0c-df4c-b8182546f6a8 6. What the following RPU runs report in XO 2nd run (00:03, VM still in that state): END failure Evacuate INTERNAL_ERROR(Object with type VM and id dd25471a-fbd3-2b0c-df4c-b8182546f6a8/config does not exist in xenopsd) END failure Rolling pool update INTERNAL_ERROR(Object with type VM and id dd25471a-.../config does not exist in xenopsd) 3rd run (00:11–00:14, after I had recovered the VM on beta only): END failure Evacuate INTERNAL_ERROR(xenopsd internal error: VM.migrate: Failed during Synchronisation point 4. msg: Xenops_interface.Xenopsd_error([S(Already_exists);[S(extra);S(extra/dd25471a-fbd3-2b0c-df4c-b8182546f6a8)]])) And on gamma for that run: 00:13:42 gamma xenopsd-xc: [debug|94|Async.VM.pool_migrate R:005a41ca1c30|xenops_utils] Key extra/dd25471a-fbd3-2b0c-df4c-b8182546f6a8 already exists 00:13:42 gamma xenopsd-xc: [error|94|...|backtrace] Raised Xenopsd_error([S(Already_exists);[S(extra);S(extra/dd25471a-fbd3-2b0c-df4c-b8182546f6a8)]]) 00:13:42 gamma xenopsd-xc: [debug|94|...|xenops_server] Caught Xenopsd_error([S(Already_exists);...]): cleaning up VM state Also at 00:14:03, opening the VM console from XO fails on beta with the same Does_not_exist … /config error. What the logs show A single transient LINSTOR controller error (ERRO:NoneType) appears to trigger the failure vdi_attach with no retry. That failure at sync point 4 leaves state on both hosts: on the source the cleanup fails with Does_not_exist …/config, and on the destination the extra/<uuid> entry stays. Subsequent RPU runs then trips on that leftover (first Does_not_exist, then Already_exists), so it can't recover by itself and requires manual recovery on both hosts. I still have the full xensource.log, SMlog and ErrorReport files for that window if someone needs more. Thanks!
  • XCP-ng 8.3 updates announcements and testing

    Pinned News
    802
    1 Votes
    802 Posts
    1m Views
    R
    Thank you, @stormi , I'm happy to know that xcp-ng.repo is not touched. These three hosts haven't been rebuilt or restored recently, no hosts have been added, and I did have to make the above modification on each one of them to get them to work. I suppose it's possible I missed the step to remove "mirrors.xcp-ng.org" from xcp-ng.repo during the initial pool build about a year ago and it's just somehow been working with our outbound Internet proxy. In any event, having "mirrors.xcp-ng.org" in our xcp-ng.repo file has definitely caused us problems in the past with our proxy setup. TGIF/DMCV!
  • 🛰️ XO 6: dedicated thread for all your feedback!

    Pinned Xen Orchestra
    326
    7 Votes
    326 Posts
    225k Views
    G
    I've noticed something that's confusing. In the Tasks tab one of my backup jobs suggests that it is only 1% done, when you expand that task then you can see that both parts are complete. Image for reference: [image: XO6backup.png] I'm guessing that if I nested other jobs, they would also display like this. I don't nest any other jobs because I want to make sure one finishes before another starts, and I don't have many VMs that I'm backing up (I don't have many VMs total). I'd request that if all nested jobs were complete, we get a full circle with 100% or just another green check mark.
  • Full backup failure when upgrading XOce

    Unsolved Backup
    2
    0 Votes
    2 Posts
    78 Views
    J
    @ph7 Thanks for sharing info. Maybe it will help someone
  • MFA for console access

    Unsolved XCP-ng
    1
    0 Votes
    1 Posts
    96 Views
    No one has replied
  • 0 Votes
    15 Posts
    724 Views
    F
    @acebmxer that is exactley what i was saying before. if you disable the remote before nas shutdown you are good on the XO side and will work indefinitely, but if you don't execute umount -l /run/xo-server/mounts/2ab02323-0d47-43a5-9081-b2454e036c70 you will have the mount stuck on the os side. You will be able to reboot XO vm eventually but you will have to wait for a long timeout during XO shutdown. if you want to simulate my issue just disable the remote 5 minutes after shutting down the synology, get the pid from service xo-server status then ps -T pid after a while you'll have 4 "rpc_wait_bit_killable" and XO will stop responding
  • Disk import not happens

    Unsolved Management
    5
    0 Votes
    5 Posts
    223 Views
    Tristis OrisT
    error via reverse proxy. Uncaught Error: Body is not a JSON, original message is : <html> <title>500 Internal Server Error</title> <style>html{color-scheme:light dark;}</style> </head> <body> <center><h1>500 Internal Server Error</h1></center> <hr><center>Angie</center> </body> </html> pr index.js:2564 me index.js:4029 i index.js:4029 promise callback*me index.js:4029 i index.js:4029 promise callback*me index.js:4029 i index.js:4029 promise callback*me index.js:4029 i index.js:4029 be index.js:4029 be index.js:4029 dr index.js:2531 importDisks index.js:2575 Lodash 2 importDisks index.js:2574 e index.js:113 P index.js:22 i index.js:22 R index.js:22 R index.js:22 import index.js:121 t factory.js:307 o factory.js:331 e action-button.js:93 m action-button.js:10 i action-button.js:10 _execute action-button.js:10 _execute action-button.js:10 _execute action-button.js:10 v action-button.js:140 componentDidMount action-button.js:147 React 14 componentWillUnmount factory.js:125 f factory.js:31 _dispatch factory.js:158 t factory.js:322 promise callback*t factory.js:313 o factory.js:331 value index.js:206 promise callback*value index.js:174 React 5
  • Booting Homeassistant (HAOS) Image breaks VM

    Unsolved Compute
    5
    0 Votes
    5 Posts
    404 Views
    C
    @IgorGlock What do you mean by "okay"? Backing up the VM with Commvault or...?
  • Rolling pool update

    Xen Orchestra
    13
    0 Votes
    13 Posts
    632 Views
    J
    @tjkreidl said: @john.c Note that a two-host pool running HA can run into the split-brain issue if one of the hosts fails; it's highly recommended to have at least three hosts in an HA pool (or look into HA-Lizard if running a pool with two hosts and HA). Right but also considering the in development Twinstor for when it goes stable and in production!
  • 4 Votes
    3 Posts
    960 Views
    J
    @vmpr Popping it behind an instance of BunkerWeb (https://www.bunkerweb.io/) or similar, with Xen Orchestra on a new IP address would help. As BunkerWeb is what’s known as a Web Application Firewall. It will help to protect your instance of Xen Orchestra, especially if you change the IP address of the appliance, and only give that friend the one that goes through BunkerWeb. As always don’t expose Xen Orchestra to the Internet, but if it’s absolutely necessary always use something like BunkerWeb (with BunkerNET) and only provide the BunkerWeb based IP address! Also run a check on all of your hosts and may be best to roll them a fresh (in case there’s an Advanced Persistent Threat). As well as a fresh instance of Xen Orchestra, and maybe encourage your friend while also doing yourself to start using hardware security keys, preferably ones that support biometrics and PINs. Alternatively other secure enclaves with cryptographic credentials held there. Additionally opt into the new encrypted datastore for Xen Orchestra on the new fresh Xen Orchestra and XCP-ng. Also make sure it’s completely up to date! Also if the hosts support it ensure that Intel TME or similar is enabled on the fresh installs of XCP-ng. Finally rotate all passwords associated with that system, including your Vates account password, especially if it’s linked to the Xen Orchestra instance hit.
  • 0 Votes
    20 Posts
    2k Views
    nathanael-hN
    @jr-m4 I did not took time to test Podman yet. But we expect this to work. You'd have to expose a socket, like the docker daemon does. Probably systemctl enable --now podman.socket would be enough. More in this page https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/building_running_and_managing_containers/using-the-container-tools-api and this one https://docs.podman.io/en/latest/markdown/podman-system-service.1.html
  • 0 Votes
    5 Posts
    545 Views
    I
    @gudge25 check these https://docs.xcp-ng.org/vms/#enabling-viridian-extensions and https://lazyadmin.nl/it/september-2026-update-break-rds-how-to-fix/