So, I recently built a XOSTOR upgrade ISO for users what had had linstor packages from 8.2 linstor-testing repositories, version which was also 1.29.2.

It might work for you.

However, I offer no guarantees. Actually, I'm not even sure that using an upgrade is the right solution for your problem. A failed update (whatever that means here) is usually recoverable from command line.

Here's the temporary link to the ISO image: https://repo.vates.tech/tmp/xcp-ng-8.3.0-20250616-linstor-upgradeonly.linstor-1-29-2.iso

SHA256 Checksum: 8a488359f64e61310ee346d2bb635f69a8ea57c76c0ee5aa2df8c419a75c7f3a

@marcoi said in XCP-ng 8.3 updates announcements and testing:

also noticed a new issue- seems like changes i had in the /etc/xensource/usb-policy.conf file for usb was lost during the upgrade.

I have some usb comm devices i use with a home assistant VM and they were gone post the upgrade.

anyway to make those change last post upgrade? Maybe make then options in gui so a config file can always be reflective of gui settings?

This one is known. I had opened an issue about it, but it didn't get much traction yet. We also have a related item in our backlog, but it's a matter of finding resources to handle it.

https://github.com/xapi-project/xen-api/issues/4935

stormi created this issue in xapi-project/xen-api

open Packaging: xapi-core RPM updates overwrite custom user config in usb-policy.conf #4935

@marcoi Let's move the discussion back there then

@shorian I might have one for you. Can you open a new thread?

@shorian Do you mean that you have hosts with XOSTOR that can't boot the installer due to broadcom drivers crashing? That's the only issue the updated ISO addresses.

@marcoi I don't have enough context to reply. You should open a new thread to discuss it, with details about your needs (always better to explain the needs before the technical solution).

Thank you everyone for your tests and your feedback!

The updates are live now: https://xcp-ng.org/blog/2025/12/18/december-2025-security-and-maintenance-updates-for-xcp-ng-8-3-lts/

@Andrew had the wrong link. I fixed it as soon as it was mentioned.

The name is xcp-ng-8.3.0-20250606.2.iso because it's the same update level as back then... Only with two updated drivers.

@dcskinner That was a mistake. Thanks for your vigilance, it's fixed!

Let us know if you have any issue with it. It's OK on our side, but I'll wait for some time before making is the new default download.

Hi! We have an updated installer for you in https://xcp-ng.org/blog/2025/12/18/december-2025-security-and-maintenance-updates-for-xcp-ng-8-3-lts/ !

I added a warning to my initial announcement.

@ovicz I'd also like to have a look at /var/log/daemon.log after a failed VM startup attempt.

@ovicz Is Secure Boot enabled on these VMs?

New security and maintenance update candidate for you to test!

A hardware issue was found in AMD Zen 5 CPU devices, related to how random numbers are generated. It's best fixed via a firmware update, but we also provide updated microcode to mitigate it, and Xen is updated to support loading the newer microcode. We also publish other non-urgent updates which we had in the pipe for the next update release.

Security updates:

• amd-microcode: This release fixes vulnerability CVE-2025-62626 in AMD Zen 5 CPUs microcode that may generate excessive number of zeros in random outputs, potentially compromising cryptographic security.
• xen:
  • Introduce support for the new Linux AMD microcode container format (multiple blobs per CPU),
  • Address the XSA-476 vulnerability (CVE-2025-58149), low severity on XCP-ng (affects an unsupported feature of Xen)
  • Enable passthrough of devices on non-zero PCI segments.
  • Improve performance of resumed or migrated VMs by supporting superpage restoration
  • Fix detection of the Self Snooping feature on capable Intel CPUs
• gpumon, xcp-featured: rebuilt for updated XAPI
• qemu:
  • Synchronize with XenServer's fix for the Windows Server 2025 NVMe write cache issue that we fixed previously
  • Fix device passthrough with devices in a PCI segment different from 0
• sm:
  • Upstream changes:
    • Robustify CBT enable/disable calls to prevent errors.
    • Various fixes regarding SCSI commands/functions.
    • Add tolerance in the GC during leaf coalesce.
    • Improves GC logging and corrects rare race conditions.
  • Our changes
    • Use serial instead of SCSI ID for SR on USB devices to prevent bad match.
    • Explicit error message during LVM metadata generation when VDI type is missing.
    • Correct and robustify LINSTOR deletion algorithm to manage in-use volumes.
    • Avoid throwing LINSTOR exceptions in case of impossible temporary volume deletion in order to properly terminate higher-level API calls.
    • Prevent XOSTOR operations if LINSTOR versions mismatches on a pool.
• varstored:
  • Restore and update the default dbx for new VMs. That's the main change for users: we now embed the latest UEFI certificates with XCP-ng, making pools ready for secure boot out of the box. We'll update the documentation to explain how to handle the transition for existing pools (ranging from "nothing to do" to "do something to ensure that future certificate updates become automatically the pool's default).
  • Fix the format of the default included KEK/db/dbx to ensure safe updates
  • Fix an issue with UEFI variable length limit
• xapi:
  • Support up to 16 VIFs (virtual network interfaces) per VM (previously: 7)
  • Runnable metrics:
    • runnable_any
    • runnable_vcpus
  • Various fixes, optimizations, small improvements, and foundational changes (such as getting prepared for a newer version of ocaml)
• gpumon xcp-featured: rebuild for updated XAPI.
• xcp-ng-pv-tools:
  • Properly detect Red Hat 10 and its derivatives, when installing the Linux guest agent
  • Update Windows Tools to 9.1.100
• xcp-ng-release: fix benign "unary operator expected" error, displayed when connecting from some terminal software
• xha: Nothing of note, minor changes such as logging typos...
• xo-lite: version 0.17.0
  • [VM/New] Fix the default topology by setting the platform:cores-per-socket value correctly (PR #9136)
  • [Host/HostSystemResourceManagement] Fix display when control domain memory is undefined (PR [#9197])
• xsconsole: Prepare for a future feature.

Optional packages updated:

• qlogic-netxtreme2-alt: alternate driver for NetXtreme2 updated to version 7.15.24.
• qlogic-qla2xxx-alt: alternate driver qla2xxx updated to version 10.02.14.01_k

Test on XCP-ng 8.3

yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-candidates
yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates
reboot

The usual update rules apply: pool coordinator first, etc.

Do not apply these updates if you are using the QCOW2 disk format. QCOW2 testing requires specific update repositories. Updating via the normal test channels would render your disks invisible, and even once the necessary packages are restored, their metadata (which disk is attached to what VM, etc.) will be lost.

For QCOW2 testers, update with:

yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates,xcp-ng-qcow2

For others who'd like to start testing with the QCOW2 format, please head towards the dedicated thread: https://xcp-ng.org/forum/topic/10308/dedicated-thread-removing-the-2tib-limit-with-qcow2-volumes

Versions:

• amd-microcode: 20251203-1.1.xcpng8.3
• gpumon: 24.1.0-71.1.xcpng8.3
• qemu: 4.2.1-5.2.15.1.xcpng8.3
• sm: 3.2.12-16.1.xcpng8.3
• varstored: 1.2.0-3.4.xcpng8.3
• xapi: 25.33.1-2.1.xcpng8.3
• xcp-featured: 1.1.8-3.xcpng8.3
• xcp-ng-pv-tools: 8.3-15.xcpng8.3
• xcp-ng-release: 8.3.0-35
• xen: 4.17.5-23.1.xcpng8.3
• xha: 25.2.0-1.1.xcpng8.3
• xo-lite: 0.17.0-1.xcpng8.3
• xsconsole: 11.0.9.1-1.1.xcpng8.3.3

Optional packages:

• qlogic-netxtreme2-alt: 7.15.24-1.xcpng8.3
• qlogic-qla2xxx-alt: 10.02.14.01_k-1.xcpng8.3

What to test

Normal use and anything else you want to test.

Test window before official release of the updates

2 days.

The installer is built. We are testing it, then we will be able to provide it to you so you can test it in turn.

We do warn about the certificate situation in the 8.3 release notes, indeed, but it's easy to get caught by that.

There's a way to temporarily disable LTS verification on the new hosts in order to join it to the existing pool.

See https://docs.xcp-ng.org/releases/release-8-3/#certificate-verification-xs which in turns points to https://docs.xenserver.com/en-us/xenserver/8/hosts-pools/certificate-verification where you'll find that command.

Regarding your initial situation, I'm not 100% sure, but I think Warm Migration, might be a way to migrate your VMs off your slave hosts while minimizing downtime. I don't know how it plays with CBT and heterogenous pool state exactly though.

@paco I think it's the first time someone asks this, which is surprising to me, because CBT enabled + local storage may not be such a rare thing.

I wasn't aware of this blocking situation. We'll need to evaluate it, document it, and if possible find a way to avoid it.

In your situation, if all you've done is upgrading the pool master, I would advise to boot the upgrade ISO again and use it to restore the 8.2 backup that was made automatically during the upgrade. Then boot the master again, disable CBT on all your disks, and start again with the upgrade.

@thomasp I'm struggling so much to find the time to do it, and my colleagues are equally busy, but I'll try to make it happen this week!