XCP-ng 8.3 updates announcements and testing

XCP-ng-JustGreat

Updated my four host home lab pool with latest batch of updates with no observed issues.

psafont

@rzr We've found a regression in the VM revert with the new version of xapi (26.1.11)

The CD VBDs of the pre-revert are not removed when reverting, this means that the reverted VM usually ends up with duplicated VBDs for CDs.

We've verified a fix for it and we're integrating it as soon as possible. This means that this test cycle will take longer.

ScarfAntennae

@rzr @stormi Perfect, thanks for clarifying!

rzr

New maintenance update candidates for XCP-ng 8.3 LTS

This release batch contains fixes, and a security fix on an optional package,

Note: the two previous batches of updates has not been released yet, so if you haven't tested it, you will see more updates than described here when you'll install the update candidates. Refer to the previous announcements.

What changed

Virtualization & System

kexec-tools: Update to sync with Xen Server:
- Add checks to reboot a crashed host if kernel crash handling doesn't complete.

Control Plane

xapi: Fix the VM revert regression introduced in earlier "testing" version.

Optional package

lldpd: Fix CVE-2026-46433, a buffer over-read when processing the "VLAN tags" from an Ethernet frame.

Versions

kexec-tools: 2.0.15-20.1.xcp-ng8.3 -> kexec-tools-2.0.15-21.1.xcpng8.3
xapi: 26.1.11-1.1.xcpng8.3 -> xapi-26.1.11-1.2.xcpng8.3

Optional packages:

lldpd: 1.0.4-1.1.xcpng8.3 -> 1.0.4-1.2.xcpng8.3

Test on XCP-ng 8.3

Warning: XOSTOR users, refer to the instructions in the previous announcement, which apply here if you haven't installed the previous update candidates.

yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-candidates
yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates
reboot

The usual update rules apply: pool coordinator first, etc.

What to test

As usual, normal use and anything else you want to test.

Test window before official release of the updates

~2 days

We would like to thank users who reported feedback since our last call for testing:
@Andrew, @ScarfAntennae, @XCP-ng-JustGreat, @acebmxer, @bufanda, @flakpyro

acebmxer

@rzr

Updates applied...

Updated:
  forkexecd.x86_64 0:26.1.11-1.2.xcpng8.3                         kexec-tools.x86_64 1:2.0.15-21.1.xcpng8.3                           
  message-switch.x86_64 0:26.1.11-1.2.xcpng8.3                    qcow-stream-tool.x86_64 0:26.1.11-1.2.xcpng8.3                      
  rrdd-plugins.x86_64 0:26.1.11-1.2.xcpng8.3                      sm-cli.x86_64 0:26.1.11-1.2.xcpng8.3                                
  squeezed.x86_64 0:26.1.11-1.2.xcpng8.3                          varstored-guard.x86_64 0:26.1.11-1.2.xcpng8.3                       
  vhd-tool.x86_64 0:26.1.11-1.2.xcpng8.3                          wsproxy.x86_64 0:26.1.11-1.2.xcpng8.3                               
  xapi-core.x86_64 0:26.1.11-1.2.xcpng8.3                         xapi-nbd.x86_64 0:26.1.11-1.2.xcpng8.3                              
  xapi-rrd2csv.x86_64 0:26.1.11-1.2.xcpng8.3                      xapi-storage-script.x86_64 0:26.1.11-1.2.xcpng8.3                   
  xapi-tests.x86_64 0:26.1.11-1.2.xcpng8.3                        xapi-xe.x86_64 0:26.1.11-1.2.xcpng8.3                               
  xcp-networkd.x86_64 0:26.1.11-1.2.xcpng8.3                      xcp-rrdd.x86_64 0:26.1.11-1.2.xcpng8.3                              
  xenopsd.x86_64 0:26.1.11-1.2.xcpng8.3                           xenopsd-cli.x86_64 0:26.1.11-1.2.xcpng8.3                           
  xenopsd-xc.x86_64 0:26.1.11-1.2.xcpng8.3                       

Complete!

Will continue to test.

flakpyro

No issues to report so far.

Andrew

@rzr Again, installed on several active pools. Running correctly for my use.

XCP-ng-JustGreat

Applied the latest batch of updates with no apparent problems.

bufanda

Applied patches yesterday, No issue seen until today.

rzr

We pushed the tested packages (of 3 batches) to the xcp-ng-updates repository, check blog post for summary and related advisories:

https://xcp-ng.org/blog/2026/06/23/june-2026-updates-2-for-xcp-ng-8-3-lts/

Thanks everyone for the tests!

marcoi

Applied the latest pushed updated to production, no major issues.
The only thing to note is my backup XCP config failed Sunday, some time post the patches being applied. The error was the Body Timeout Error. This morning the backup ran without issues. Not sure why, but it wasn't a big deal and it working now. So not sure tied in with patching or not.

rzr

@marcoi said:

Applied the latest pushed updated to production, no major issues.

Thank you for testing

The only thing to note is my backup XCP config failed Sunday, (...) This morning the backup ran without issues.

Glad it has finally to work, if problem occurs again feel free to ask about this feature in this subforum:

https://xcp-ng.org/forum/category/21/backup

rzr

XCP-ng 8.3 LTS Updated installer ISO

The installer ISO has been refreshed with up date packages already published.

The image can be downloaded from:

https://repo.vates.tech/tmp/xcp-ng-8.3.0-20260709.testing.iso

The procedure to install XCP-ng 8.3 remains the same.

The refreshed installer ships updated Xen, Linux kernel and some drivers/firmware, which help with deploying XCP-ng 8.3 on hardware that needs changes introduced by updates not present in previous ISO image releases.

This refreshed ISO has been validated on a subset of supported hardware. Additional testing on any hardware from the community would be greatly appreciated.

As with previous installer releases, it can be used to upgrade XCP-ng from a previous major release (8.0, 8.1 or 8.2). Refer to the Upgrade documentation. For existing XCP-ng 8.3 installations, there’s no need to use this ISO image. Regular yum updates achieve the same results as a fresh installation with that ISO image.

Installer improvements include minor enhancements related to software RAID1 support for the system disks, including support for more than 2 disks in the RAID1 array if anyone really wants that.

New maintenance update candidates for XCP-ng 8.3 LTS

This release batch includes a VBD leak regression fix in XAPI and minor changes needed to build the installer ISO.

A regression was introduced to the VM.revert operation in xapi-26.1.4-3.3, which left VBDs attached to the VM when they are not present in the reverted-to snapshot (instead of destroying such VBDs, as was done before).

VMs that were reverted to their snapshots after xapi-26.1.4-3.3 may have such "leftover" VBDs. Advanced users can use the leaked_vbds script to detect possible cases in your pools. Sadly, there is no way to automatically distinguish a VBD that was created after a VM.revert from a VBD that survived the VM.revert, so this script relies on the user to determine if the suspicious VBDs are indeed affected or not.

What changed

Control Plane

xapi: Fix non-snapshotted VBDs staying attached after VM.revert

Versions

xapi: 26.1.11-1.2.xcpng8.3 -> 26.1.11-1.3.xcpng8.3
xcp-ng-release: 8.3.0-37 -> 8.3.0-38

Test on XCP-ng 8.3

yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-candidates
yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates
reboot

The usual update rules apply: pool coordinator first, etc.

What to test

As usual, normal use and anything else you want to test.

Test window before official release of the updates

~5 days

We would like to thank users who shared feedback since our last call for testing: @Andrew, @XCP-ng-JustGreat, @acebmxer, @bufanda, @flakpyro , @marcoi.

acebmxer

@rzr

Just tried to apply update. Unable to put master host into maintenance mode - not enough memory...

https://xcp-ng.org/forum/post/106814

Edit - Updated both host after manually migrating the vms. Also noticed when i thought i was updating the master host first it might not have been.

With Twinstore installed and enabled HA was also enabled. After first host came back on line i looked at the pool advanced settings and saw host 2 was master not host 1.

Both host have been updated and rebooted. @olivierlambert would this change post master from one host to the other? XO did loose access to the VMs while the first host rebooted. So i am alittle confused...

Once host 2 came back online pool master is switched back to host 1 again...

will look for any other odd things.

flakpyro

@rzr

Is there something specific needed for the leaked_vbds script?

When i run it i get:

[09:37 xcpng-prd-01 ~]# ./leaked_vbds 
Traceback (most recent call last):
  File "./leaked_vbds", line 23, in <module>
    snap = xapi.VM.get_parent(snap)
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 317, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 198, in xenapi_request
    result = _parse_result(getattr(self, methodname)(*full_params))
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 292, in _parse_result
    raise Failure(result['ErrorDescription'])
XenAPI.Failure: ['HANDLE_INVALID', 'VM', 'Ref:29']

andriy.sultanov

@flakpyro I'll add some additional error-checking, but looks like you're affected by the snapshot_of issue and should run another script (yes, I know...) to fix these stale references: https://github.com/xcp-ng/xcp/blob/master/scripts/snapshot-fixer.py

flakpyro

@andriy.sultanov

Thanks

I ran that second script on the master and it did indeed find entries:

./snapshot-fixer.py rewrite
INFO:root:Check HA...
INFO:root:Shutting down xapi...
INFO:root:Regenerating database...
INFO:root:The VM d6befe36-fea7-04f3-25f1-feada684702b has Ref:9 as its "snapshot_of" value, changing to null.
INFO:root:The VM e013d9ed-63b1-9c9f-a228-d48a6047688c has Ref:89 as its "snapshot_of" value, changing to null.
INFO:root:The VM 060df7ee-8af4-4c93-8265-bfa3024ccacd has Ref:1 as its "snapshot_of" value, changing to null.
INFO:root:The VM da7b6ff6-d722-2120-7cc4-a1e0038a216e has Ref:307 as its "snapshot_of" value, changing to null.
INFO:root:The VM 4178b057-efee-427e-bcad-67323419ac7a has Ref:9 as its "snapshot_of" value, changing to null.
INFO:root:The VM 8d624e49-ca99-4af6-b17c-38cad0646194 has Ref:1 as its "snapshot_of" value, changing to null.
INFO:root:The VM dc72da43-f144-3343-5ee5-a0dbffd209f6 has Ref:251 as its "snapshot_of" value, changing to null.
INFO:root:The VM e8c6376b-49e1-422b-551a-35420f70af2a has Ref:8 as its "snapshot_of" value, changing to null.
INFO:root:The VM 25eb14ab-e943-0d18-2137-c681e1191346 has Ref:11 as its "snapshot_of" value, changing to null.
INFO:root:The VM 6f819166-b008-4a03-a692-fcbd75018be9 has Ref:1 as its "snapshot_of" value, changing to null.
INFO:root:The VM 9d1a57e8-3531-4b55-9682-f775215866df has Ref:1 as its "snapshot_of" value, changing to null.
INFO:root:The VM 0ac88f7c-d79b-281e-cf1f-a2392ba594ed has Ref:29 as its "snapshot_of" value, changing to null.
INFO:root:The VM 1f3541c4-8c81-4959-9cbe-edbec9481220 has Ref:1 as its "snapshot_of" value, changing to null.
INFO:root:Writing database to /var/lib/xcp/state.db
INFO:root:Starting up xapi...

after which it shows clean now:

./snapshot-fixer.py dry-run
INFO:root:Regenerating database...

However i am still getting the same error when trying to run leaked_vbs:

./leaked_vbds 
Traceback (most recent call last):
  File "./leaked_vbds", line 23, in <module>
    snap = xapi.VM.get_parent(snap)
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 317, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 198, in xenapi_request
    result = _parse_result(getattr(self, methodname)(*full_params))
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 292, in _parse_result
    raise Failure(result['ErrorDescription'])
XenAPI.Failure: ['HANDLE_INVALID', 'VM', 'Ref:29']

Andrew

@rzr Updates installed and running. I also run the snapshot-fixer.py and xapi_leaked_vbds.py scripts... The fixer found/fixed lots on many pools...

@andriy.sultanov I have the same error... On one pool, I ran snapshot-fixer.py and it fixed about 50 of them, but xapi_leaked_vbds.py still returns an error (before and after fix):

Traceback (most recent call last):
  File "./xapi_leaked_vbds.py", line 23, in <module>
    snap = xapi.VM.get_parent(snap)
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 317, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 198, in xenapi_request
    result = _parse_result(getattr(self, methodname)(*full_params))
  File "/usr/lib/python3.6/site-packages/XenAPI.py", line 292, in _parse_result
    raise Failure(result['ErrorDescription'])
XenAPI.Failure: ['HANDLE_INVALID', 'VM', 'Ref:119']

andriy.sultanov

@Andrew @flakpyro I've updated the script, could you try running it again?

https://gist.github.com/last-genius/edf7dab3b1847ec5bb1d9917996eba41

Andrew

@andriy.sultanov No error... returns nothing.