XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XCP-ng 8.3 updates announcements and testing

    Scheduled Pinned Locked Moved News
    805 Posts 66 Posters 1.1m Views 82 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      robertblissitt
      last edited by robertblissitt

      I did a RPR early this morning just to start fresh and then kicked off an RPU 15 minutes ago. The RPU evacuated all the VMs from the master successfully but the RPU is stuck at 0% (as seen in Pool Tasks). I will allow the process to time out (or restart XO if the wait is too long), promote another node to master, and retry the RPU.

      vxgmichelV 1 Reply Last reply
      Reply Quote 1
      • G
        Greg_E
        last edited by

        Did my 3 host production pool, only glitches were a few users that were logging in right as vms were migrating. Had a compressed day and had to just Yolo this morning instead of waiting until slower times. Getting paranoid about exploits, and it was Windows patch day too, so just jammed a bunch of stuff together this morning.

        1 Reply Last reply
        Reply Quote 2
        • vxgmichelV
          vxgmichel Vates 🪐 XCP-ng Team @Andrew
          last edited by

          @Andrew said:

          Waiting before re-enabling the load balancer task is stuck waiting...

          Was it stuck for longer than 30 minutes? If it simply appeared in the task list with a pending status for less than 30 minutes that's actually expected. I ran an RPU on some hosts of my own and made a screenshot for comparison, you can see how the task should look when it's pending and when it's completed.

          image.jpeg

          This task has been introduced in XO 6.7.0 (2026-07-30) by the way, so it's fairly new:

          [RPU] Re-enable the load balancer after a configurable safe delay (30 minutes by default) when a rolling pool update ends (PR #10111)

          Please let us know if you notice something wrong with it.

          1 Reply Last reply
          Reply Quote 0
          • vxgmichelV
            vxgmichel Vates 🪐 XCP-ng Team @robertblissitt
            last edited by vxgmichel

            @robertblissitt said:

            The RPU evacuated all the VMs from the master successfully but the RPU is stuck at 0% (as seen in Pool Tasks).

            Thanks for the report, we have an item in our issue tracker that matches your description. It should be fixed in the upcoming 26.1.23-2.1 xapi version that will be part of the next maintenance update. Please let us know if it keeps happening after the next maintenance update.

            R 1 Reply Last reply
            Reply Quote 0
            • julienXOvatesJ
              julienXOvates Vates 🪐 XO Team @Andrew
              last edited by

              @Andrew could you send us logs over a support ticket so we can understand why the RPR failed ? Thanks

              1 Reply Last reply
              Reply Quote 0
              • R
                robertblissitt @vxgmichel
                last edited by

                @vxgmichel I may have found our problem. Looks like an XCP-ng update may have slipped an unwanted site back into our /etc/yum.repos.d/xcp-ng.repo. When I first deployed XCP-ng, I had to remove the "http://mirrors.xcp-ng.org/8/8.3/base/x86_64/" entry for updates to work as it was too troublesome to get it working through our outbound proxy managed by another team. I have re-removed that site and things are working again. It was not fun trying to figure that out in the midst of a vulnerability.

                diff /etc/yum.repos.d/xcp-ng.repo /etc/yum.repos.d/xcp-ng.repo.orig
                <- baseurl=http://updates.xcp-ng.org/8/8.3/base/x86_64/
                -> baseurl=http://mirrors.xcp-ng.org/8/8.3/base/x86_64/ http://updates.xcp-ng.org/8/8.3/base/x86_64/

                <- baseurl=http://updates.xcp-ng.org/8/8.3/updates/x86_64/
                -> baseurl=http://mirrors.xcp-ng.org/8/8.3/updates/x86_64/ http://updates.xcp-ng.org/8/8.3/updates/x86_64/

                stormiS 1 Reply Last reply
                Reply Quote 2
                • stormiS
                  stormi Vates 🪐 XCP-ng Team @robertblissitt
                  last edited by

                  @robertblissitt said:

                  Looks like an XCP-ng update may have slipped an unwanted site back into our /etc/yum.repos.d/xcp-ng.repo.

                  Are you sure that this is what happened? XCP-ng updates do not overwrite this file if it has local modifications. It's defined as %config(noreplace) in the RPM spec file for package xcp-ng-release. And no update scriptlets edit it either.

                  Plus, config files are only updated by rpm when they change, and the file hasn't changed since May 2023.

                  A few other hypotheses towards the cause:

                  • an upgrade or reinstall via ISO image
                  • a host that never had the edit, added to the pool later
                  • a provisioning tool
                  R 1 Reply Last reply
                  Reply Quote 1
                  • R
                    robertblissitt @stormi
                    last edited by

                    Thank you, @stormi , I'm happy to know that xcp-ng.repo is not touched. These three hosts haven't been rebuilt or restored recently, no hosts have been added, and I did have to make the above modification on each one of them to get them to work. I suppose it's possible I missed the step to remove "mirrors.xcp-ng.org" from xcp-ng.repo during the initial pool build about a year ago and it's just somehow been working with our outbound Internet proxy. 🙂 In any event, having "mirrors.xcp-ng.org" in our xcp-ng.repo file has definitely caused us problems in the past with our proxy setup.

                    TGIF/DMCV!

                    1 Reply Last reply
                    Reply Quote 1
                    • vxgmichelV
                      vxgmichel Vates 🪐 XCP-ng Team
                      last edited by

                      New maintenance update candidates for XCP-ng 8.3 LTS

                      This batch of updates contains security fixes, new Windows guest tools, and other improvements. It also changes some defaults of the SSH server: read the OpenSSH section before you update.

                      What changed

                      Virtualization

                      • guest-templates-json:
                        • Sync with XenServer guest-templates-json-2.0.17-1.
                        • New guest template: Ubuntu Resolute Raccoon 26.04.
                        • The Rocky Linux 10 template does not change (15 GB default disk).
                      • swtpm: This release contains a fix for CVE-2026-75900, an out-of-bounds read in the swtpm daemon that could cause a crash of the daemon.
                      • xcp-ng-pv-tools:
                        • Update to XCP-ng Windows Guest Tools 9.2.445.
                        • Remove the XenTools-fix-9.2.351.msp hotfix tool for users of 9.2.350.
                      • xen: This release is a synchronization with XenServer release 4.17.7-6. It contains the patches for XSA-524 and XSA-525, and a fix for lost timer interrupts. It also fixes a bug in the original XSA-524 fix where the IOMMU device table size on AMD systems was always set to 2 MiB.

                      System

                      • gnutls: Update to gnutls 3.8.12. This update fixes multiple vulnerabilities, including CVE-2026-3833.
                      • libssh2: Update libssh2 to fix five security vulnerabilities. They affect the SFTP/SCP connections that curl and Midnight Commander (mc) make:
                        • CVE-2026-58051: potential arbitrary free when listing public keys
                        • CVE-2026-66032: use-after-free in SFTP
                        • CVE-2026-66033: out-of-bounds read/write with the AES-GCM cipher
                        • CVE-2026-66034: out-of-bounds read when listing public keys
                        • CVE-2026-66035: heap buffer overflow during ETM (Encrypt-then-MAC) decryption
                      • nettle: Update to nettle 3.10. Sync with XenServer nettle-3.10-3, which includes a fix that XCP-ng released previously.
                      • openssh: Update to OpenSSH 9.9p1. This update fixes 21 CVEs: CVE-2026-3497, CVE-2026-35386, CVE-2026-35387, CVE-2026-55653, CVE-2026-55654, CVE-2026-55655, CVE-2026-59995, CVE-2026-59996, CVE-2026-59997, CVE-2026-59999, CVE-2026-60000, CVE-2026-60001, CVE-2026-60002, CVE-2026-73281, CVE-2026-73282, CVE-2026-73283, CVE-2026-106552, CVE-2026-106553, CVE-2026-106582, CVE-2026-106583, CVE-2026-106585.
                        • Post-Quantum Key Exchange is now available. It is the preferred key exchange by default.
                        • FIDO2 keys are now available, but they are disabled by default. See Enabling FIDO2 keys.
                        • The SSH client now reads drop-in files in /etc/ssh/ssh_config.d/.
                        • Fix a temporary lockout with ssh-copy-id.
                        • We hardened the SSH server configuration with our security team. Some changes modify how existing configurations work:
                          • sshd refuses SSH user certificates by default. If you log in with certificates, follow SSH user certificates before the update, or you will lose SSH access.
                          • Forwarding is limited to the host itself (VNC console and XO Lite/XAPI tunnels continue to work). See Forwarding.
                          • MaxAuthTries is now 3, LoginGraceTime is now 60 seconds, and PerSourcePenalties (brute-force protection) is enabled. See What you can safely customize.
                          • sshd no longer accepts hmac-sha1 and ssh-rsa (SHA-1). Very old SSH clients can fail to connect.
                          • The update restarts sshd and replaces /etc/ssh/sshd_config. Put your changes in drop-in files: see Do not edit our configuration files.
                      • openssl: Update to openssl 3.5.7. This is a security update that fixes multiple CVEs, including one rated High (CVE-2026-45447, heap use-after-free in PKCS7_verify()). It also contains a fix for CVE-2026-72897. See the openssl vulnerabilities page for more information.
                      • samba: Sync with XenServer: update samba to 4.23.3, to support LDAPS and the Windows security hardening.
                      • xcp-ng-deps: Remove ldns and unbound-libs as they are not used (this avoids the frequent security updates of these packages).
                      • New and updated dependencies of gnutls 3.8, samba 4.23 and OpenSSH FIDO2 support: libcbor, libfido2, libidn2, libtalloc, libtdb, libtevent. Rebuilds with the updated gnutls: wget, and the optional neon, nut and nbdkit.

                      Control Plane

                      • xapi:
                        • Fix an RPU that could stop in some cases, because xapi lost track of an event.
                        • Improve performance for storage migrations with snapshots, with a fix of the migration order.
                        • Support adding SSL certificates signed with SHA384.
                        • Fix SM plugins that xapi registered twice. This broke SR functions.
                        • Fix migration of VDIs with tags to SRs that are not plugged on the source host.
                        • Fix incomplete VDI cleanup when HA is disabled while a host is fenced.
                        • Fix a host that could stop when it joins the pool, because of a dangling reference.
                        • sdn-controller: Add reliability fixes, make sure that xapi applies the rules.
                        • This update contains many storage-related changes. We tested each change alone, but not all together. Please test your storage operations.
                      • xcp-python-libs: Sync with XenServer xcp-python-libs-3.0.11-1.
                      • xha: Fix a potential memory access crash during the activation step.

                      Storage

                      • blktap: This release contains 2 performance improvements and 2 important fixes:
                        • The NBD server of tapdisk does fewer memory allocations. This improves VDI migrations.
                        • The polling algorithm of tapdisk sends fewer notifications. This improves the performance in polling mode.
                        • Fix a possible VDI inconsistency with Windows VMs, when the Windows driver reconnects a stalled ring.
                        • Fix a possible local data corruption during a storage migration with CBT enabled.
                      • nbd: Update to nbd 3.25 (sync with XenServer).
                      • sm:
                        • Fix LINSTOR VDI locking failing during a transient state, which could prevent a user from executing a VDI command.
                        • LINSTOR SR scans are now ~35% faster (e.g. ~1m08s down to 44s on a host with 950 VDIs in our tests).
                        • Fix diskless LINSTOR resources being cleaned up even though they are still in use.
                        • Fix an issue on LVM SRs that could block VDI activation of QCOW2 VDIs on a non-coordinator host.
                        • Fix incremental backup failures (SR_BACKEND_FAILURE_460) with CBT on file-based SRs.
                        • Clean up gc_running_file on GC exception so that operations on the SR can continue after the GC error.

                      Network

                      Optional packages

                      • libreswan: Remove DNSSEC support. This allows us to remove ldns and avoid the frequent security updates of this package.

                      UI

                      • xo-lite:
                        • [New network] Choose between no NBD, secure NBD and insecure NBD (not encrypted) when creating a network (PR #10540)
                        • [VM/System] Fix video RAM displayed in bytes instead of MiB (PR #10486)
                        • [Treeview] Add a search bar and virtualize the tree (PR #10414)
                        • [New/VM] Hide guest tools ISO SR in new VM ISO selector (PR #10430)
                        • [Host] Add possibility to enable/disable a host (PR #10337)
                        • [Host] Add possibility to restart a host toolstack (PR #10480)
                        • [Host] Add possibility to detach a host (PR #10520)
                        • [Side panels] Harmonize card titles across side panels (PR #10289)
                        • [VM/New] Fix UEFI VMs booting in BIOS mode when created with a network install or without disk (PR #10521)
                        • [Treeview] Press Ctrl+K (Cmd+K on macOS) to focus the treeview search (PR #10492)
                        • [Pool] Fix connection, stats and consoles when overriding the pool master with ?master=: use HTTPS by default instead of the protocol and port of the page (PR #9955)
                        • [Pool/Host/VM] Add copy button for multiple information on Host/VM Dashboard and Pool/Host/VM System tabs (PR #9976)
                        • [New/VM] Fix MAP_DUPLICATE_KEY error that could randomly prevent the VM creation (PR #10552)
                        • [Host/Network] Add the possibility to forget a PIF (PR #10239)
                        • [Lite] Display dates and times as YYYY-MM-DD HH:mm:ss (PR #10547)
                        • [Pool/Hosts] Add a side panel displaying the selected host's details, and a search bar to filter the hosts table (PR #10580)
                        • [VM/Host] Add console clipboard management (PR #10625)
                        • [VM/Network] Add the possibility to edit a VIF (PR #10505)

                      Drivers and Firmware

                      • amd-microcode: Update to the 2026-10-01 AMD microcode drop as redistributed by XenServer. Updated CPUs:
                        • GN-B1 00a00f11: 2025-04-18, rev 0a0011de -> 2026-06-19, rev 0a0011e1
                        • GN-B2 00a00f12: 2025-03-27, rev 0a001247 -> 2026-06-19, rev 0a00124e
                        • RS-B1 00a10f11: 2025-06-09, rev 0a101158 -> 2026-07-06, rev 0a10115b
                        • RS-B2 00a10f12: 2025-06-12, rev 0a101253 -> 2026-07-01, rev 0a10125b
                        • RSDN-A2 00aa0f02: 2025-06-12, rev 0aa0021c -> 2026-07-06, rev 0aa0021f
                      • cisco-enic: Update to 4.7.0.6 (sync with XenServer).
                        • Add SR-IOV v2 support with PF/VF mailbox communication and lifecycle management.
                        • Add administrative queue support and improve queue/IRQ handling.
                        • Improve PF/VF link-state, MTU, shutdown, and SR-IOV teardown handling.
                        • This is a large update of the driver. If you have Cisco hardware that uses enic, please test it.
                      • intel-microcode: This update installs the Intel IPU 2026.4 microcode from the XenServer 20260929-1 release.
                      • mellanox-mlnxen: Fix ConnectX-4 Ethernet and RDMA devices that the driver did not register correctly.

                      Optional packages

                      • r8125-module-alt: New alternative driver (Realtek 9.018.00). Various bug fixes for 2.5G Realtek NICs. It does not support the 5G devices: use the main driver for them.

                      Versions

                      • amd-microcode: 20260923-1.1.xcpng8.3 -> 20261001-2.1.xcpng8.3
                      • blktap: 3.55.5-12.3.xcpng8.3 -> 3.55.5-12.4.xcpng8.3
                      • cisco-enic: 4.5.0.7-1.xcpng8.3 -> 4.7.0.6-1.1.xcpng8.3
                      • gnutls: 3.3.29-10.2.xcpng8.3 -> 3.8.12-1.1.xcpng8.3
                      • gpumon: 24.1.0-99.1.xcpng8.3 -> 24.1.0-100.1.xcpng8.3
                      • guest-templates-json: 2.0.16-1.1.xcpng8.3 -> 2.0.17-1.1.xcpng8.3
                      • intel-microcode: 20260925-1.xcpng8.3 -> 20260929-1.xcpng8.3
                      • libcbor: 0.11.0-3.1.xcpng8.3
                      • libfido2: 1.14.0-7.1.xcpng8.3
                      • libidn2: 2.3.4-4.xcpng8.3
                      • libssh2: 1.11.0-1.xcpng8.3 -> 1.11.0-1.2.xcpng8.3
                      • libtalloc: 2.1.16-1.el7 -> 2.4.3-1.1.xcpng8.3
                      • libtdb: 1.3.18-1.el7 -> 1.4.14-1.1.xcpng8.3
                      • libtevent: 0.9.39-1.el7 -> 0.17.1-1.xcpng8.3
                      • mellanox-mlnxen: 5.9_0.5.5.0-3.1.xcpng8.3 -> 5.9_0.5.5.0-3.2.xcpng8.3
                      • nbd: 3.24-1.xcpng8.3 -> 3.25-2.1.xcpng8.3
                      • nettle: 2.7.1-9.el7_9 -> 3.10-3.1.1.xcpng8.3
                      • openssh: 9.8p1-1.2.6.xcpng8.3 -> 9.9p1-33.2.xcpng8.3
                      • openssl: 3.5.5-1.3.xcpng8.3 -> 3.5.7-1.1.xcpng8.3
                      • rpm: 4.11.3-32.el7 -> 4.11.3-32.1.xcpng8.3
                      • samba: 4.10.16-25.3.xcpng8.3 -> 4.23.3-1.1.xcpng8.3
                      • sm: 3.2.12-25.2.xcpng8.3 -> 3.2.12-25.3.xcpng8.3
                      • swtpm: 0.7.3-12.1.xcpng8.3 -> 0.7.5-1.xcpng8.3
                      • wget: 1.21.4-1.1.xcpng8.3 -> 1.21.4-1.2.xcpng8.3
                      • xapi: 26.1.21-1.4.xcpng8.3 -> 26.1.23-2.1.xcpng8.3
                      • xcp-featured: 1.2.1-6.xcpng8.3 -> 1.2.1-7.xcpng8.3
                      • xcp-ng-deps: 8.3-14 -> 8.3-16
                      • xcp-ng-pv-tools: 8.3-19.xcpng8.3 -> 8.3-21.xcpng8.3
                      • xcp-python-libs: 3.0.10-1.1.xcpng8.3 -> 3.0.11-1.1.xcpng8.3
                      • xen: 4.17.7-4.2.xcpng8.3 -> 4.17.7-6.2.xcpng8.3
                      • xha: 25.2.0-1.1.xcpng8.3 -> 25.2.0-1.2.xcpng8.3
                      • xo-lite: 0.26.0-1.xcpng8.3 -> 0.27.0-1.xcpng8.3

                      Optional packages

                      • fakechroot: 2.20.1-2.el7
                      • http-nbd-transfer: 1.7.0-1.xcpng8.3 -> 1.8.0-1.xcpng8.3
                      • libreswan: 4.12-2.3.3.xcpng8.3 -> 4.12-2.3.4.xcpng8.3
                      • nbdkit: 1.8.0-4.el7 -> 1.8.0-4.1.xcpng8.3
                      • neon: 0.30.0-3.el7 -> 0.30.2-6.1.xcpng8.3
                      • nut: 2.8.0-2.1.xcpng8.3 -> 2.8.0-2.2.xcpng8.3
                      • python3-py: 1.4.32-2.el7
                      • python3-pytest: 2.9.2-5.el7
                      • r8125-module-alt: 9.018.00-1.xcpng8.3

                      Test on XCP-ng 8.3

                      Warning: if you log in to your hosts with SSH user certificates, or if you edited /etc/ssh/sshd_config, read the OpenSSH section before the update.

                      yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-candidates
                      yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates
                      reboot # required: new Xen hypervisor, microcode, and drivers
                      

                      The usual update rules apply: pool coordinator first, etc.

                      What to test

                      As usual, normal use and anything else you want to test. These checks help us most:

                      1. SSH: deploy a public key with ssh-copy-id, then connect with the key. It must work. If you use SSH tunnels (VNC console, XO Lite) or FIDO2 keys, test them too.
                      2. Storage: enable CBT on the disks of a running VM, migrate the VM with its storage, then run an incremental backup. Also do your usual backups, restores and snapshots. They must complete, and the data must be correct.
                      3. If you have Cisco hardware that uses the enic driver, make sure that the NICs have link and pass traffic. If you use SR-IOV on them, test it too.
                      4. If you use XOSTOR, do SR scans and VDI operations. They must complete without errors.

                      If a check fails, tell us the hardware, the VM type, and provide the logs (/var/log/xensource.log, /var/log/SMlog, /var/log/secure).

                      Test window before official release of the updates

                      ~6 days

                      We would like to thank users who shared feedback since our last call for testing:
                      @acebmxer, @Andrew, @XCP-ng-JustGreat, @JeffBerntsen, @hoh, @flakpyro, @robertblissitt, @Greg_E

                      F acebmxerA 2 Replies Last reply
                      Reply Quote 0
                      • F
                        flakpyro @vxgmichel
                        last edited by

                        Installed on my usual test systems without issue. (AMD and Intel)
                        Ran a backup on a pool not using CBT, no issue.
                        Ran a backup on a pool that uses CBT, no issues.
                        SSH public key auth continues to work as before.

                        1 Reply Last reply
                        Reply Quote 0
                        • acebmxerA
                          acebmxer @vxgmichel
                          last edited by

                          @vxgmichel

                          Big update...
                          Installed on Intel 13th gen and 14gen cpus

                          CBT was already enabled on vms prior to update.
                          2026-11-19T19_42_40.762Z - backup NG.json.txt - After update - regular incremental backup

                          2026-11-19T19_46_25.835Z - backup NG.json.txt Migrated 1 vm from nvme SR QCOW2 format to hdd SR QCOW2 format and did another incremental backup.

                          1 Reply Last reply
                          Reply Quote 0

                          Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                          Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                          With your input, this post could be even better 💗

                          Register Login
                          • First post
                            Last post