XCP-ng 8.3 updates announcements and testing

[acebmxer]

@rzr

Edit - If @olivierlambert wants to make this post its own thread im ok with that.

Updated home lab and live convert to qcow2 seems to work.

Process was a little long 30ish min. but it worked and did not fail.

Kubuntu 26.04 lts beta

Tried a windows vm got error - 2026-04-16T13_04_47.720Z - XO.txt

Gives VDI has CBT enabled...

Windows 11 vm.

From backup job...

Just tried another vm that was powerd off. regular ubuntu 24.04 successfull...

[dthenot]

@acebmxer Hello,

The error VDI_CBT_ENABLED means that the XAPI doesn't want to move the VDI to not break the CBT chain.
You can disable the CBT on the VDI before migrating the VDI but if you have snapshots with CBT enabled it can be complicated and it might necessitate to remove them before moving the VDI.
We have changes planned to improve the CBT handling in this kind of case.

[dthenot]

@Andrew Hello Andrew,

Thank you for reporting.
It appear that the CBT on FileSR-based SR is not working in addition to data-destroy (the option that allow to remove the VDI content and only keep the CBT).
Can you confirm that you are using a FileSR (ext or nfs)?
Is it possible to disable purge data on the CR job?

[Andrew]

@dthenot Yes, VHD files on both local disk and NFS, same problem.

Testing one VM, I removed the snapshot, disk CBT setting, and removed the destination replica. First CR run does a full backup without issue (same NBD/CBT/purge enabled). Second run has the same problem (fell back to full). So, clearing out things does not fix it (with the same original setup).

Testing several combinations, just disabling the backup purge snapshot option makes the delta CR backup work again (NBD/CBT still enabled). It does a full backup the first run (fell back to full), but then does delta after that.

[acebmxer]

@dthenot

I have been able to migrate all vms over to qcow2. Think shutting down the vms and booting backup. Alos if anything from this thread might have had an impact. https://xcp-ng.org/forum/topic/12087/backups-with-qcow2-enabled/9

[dthenot]

@Andrew Hello,

I have been able to find the problem and make a fix, it's in the process of being packaged.
I can confirm it only happen for file based SR when using purge snapshots.
For some reason, the vdi type of CBT_metadata is cbtlog for FileSR but stays the image format it was for LVMSR
And it would make a condition fail during the list_changed_blocks call.

[olivierlambert]

Nice catch @dthenot !

[Andrew]

@dthenot Great! I'm happy I was able to help test it. I look forward to the update release.

Interesting note, CR is faster when the snapshots are not deleted.... or CR is faster because of the update, I'll test again after the fix.

[rzr]

Feature fixes, security and maintenance update candidates for you to test!

This release batch contains fixes on the major storage feature previously announced,
read the RC2 announcement for QCOW2 image format support for 2TiB+ images.

The whole platform has been hardened with back-porting security patches from the latest version of OpenSSH.

An additional driver fix is part of this minor package set.

What changed

Storage

QCOW2 image format support is the major feature of this release batch,
check related announcement in forum.

Some fixes have been applied to fix issues found during the testing phase. Many thanks go to @Andrew who found a CBT-related bug on file-based SRs!

• sm: 3.2.12-17.5
  • Fix a regression on CBT (Changed block tracking) on file-based SRs (EXT, NFS, ...), causing backup jobs using the "purge snapshot data when using CBT" option to create full backups each time instead of deltas.
  • Deactivate unused LVM snapshot base before deletion to prevent LVM leak. This fix is not related to the QCOW2 feature, but is important and localized enough for us to provide it in addition the other changes.
  • Minor fix that prevents a warning when updating the package.
• blktap: 3.55.5-6.5
  • Fix install warning when triggering mdadm to generate a udev rule.

Network

• openssh: Update to 9.8p1-1.2.3
  • Two vulnerabilities disclosed along with the OpenSSH 10.3 release have been fixed.
    • In authorized_keys, when principals="" was defined along with a CA with a common CA, an interpretation error occurred, which could lead to unauthorized access.
    • When one ECDSA algorithm was active, it activated all others regardless of their configuration. (By default, all ECDSA algorithms are active.)
  • For more details please track the upcoming Vates Security Advisories.

Drivers updates

More information about drivers and current versions is maintained on the drivers wiki page.

• qlogic-fastlinq-alt: 8.74.6.0-1
  • Fixes 2 issues in the qede module driver:
    • Driver does not retain configured MAC and MTU post reset recovery
    • Driver does not recover from TX timeout error

Versions:

• blktap: 3.55.5-6.4.xcpng8.3 -> 3.55.5-6.5.xcpng8.3
• openssh: 9.8p1-1.2.2.xcpng8.3 -> 9.8p1-1.2.3.xcpng8.3
• sm: 3.2.12-17.2.xcpng8.3 -> 3.2.12-17.5.xcpng8.3

Optional packages:

• qlogic-fastlinq-alt: 8.70.12.0-1.xcpng8.3 -> 8.74.6.0-1.xcpng8.3

Test on XCP-ng 8.3

If you are using XOSTOR, please refer to our documentation for the update method.

yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-candidates
yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates
reboot

The usual update rules apply: pool coordinator first, etc.

What to test

The most important change is related to storage: adding QCOW2 support also affects the codebase managing VHD disks. What matters here is, above all, to detect any regression on VHD support (we tested it deeply, but on this matter there's no such thing as too much testing). Of course, you are also welcome to test the QCOW2 image format support.

See the dedicated thread for more information.

Other significant changes requiring attention:

• SSH connectivity

And, as usual, normal use and anything else you want to test.

Test window before official release of the updates

~4 days

We would like to thank users who reported feedback on the QCOW RC2 release: @acebmxer, @andrew, @bufanda, @flakpyro, @jeffberntsen, @ph7

[acebmxer]

@rzr

installed updates will report back.

Update - I had migrated vms back over to vhd prior to update release. I have migrated 2 vms back over to qcow2 and the initial backup ran successfull. Ran a second delta backup and that as well was successful with out issues. Backups happen very quickly now. But it appears the % and progress bar are working.

When CBT is enabled on the vm vdi. They show up as needing to be coalesced. VMs without CBT enabled the vdis are coalesced.

Will continue to monitor.