XCP-ng 8.3 updates announcements and testing

[archw]

@Andrew
Thanks!

[stormi]

@archw It's written in the blog post

[archw]

@stormi I missed it!

[gduperrey]

New update candidates for you to test!

As we move closer to making XCP-ng 8.3 the new LTS release, taking over from XCP-ng 8.2.1, a first batch of updates is now available for user testing ahead of a future collective release. Details are provided below.

• amd-microcode: Packaging and versioning update, but no actual changes in microcodes.
• blktap: Fixes.
• broadcom-bnxt-en: bug fix: "Backport patch to fix GSO type for HW GRO packets on 5750X chips"
• busybox: backport fixes for CVE-2018-20679 and others.
• gpumon: No major changes. Rebuild for dependency reasons.
• guest-templates-json: Add templates for Windows server 2025 and Ubuntu 24.04. Remove "preview" from a few template names.
• host-upgrade-plugin: Update to version 3.0.1 which transitions to python 3 and brings some fixes.
• intel-i40e: Update to version 2.25.11
• interface-rename: Sync with XenServer, but this only changes packaging details.
• ipxe: Rebuild.
• jemalloc: Updated to version 5.3.0.
• lvm2: Fixes.
• microsemi-smartpqi: Update to version 2.1.30_031
• ncurses: Updated to upstream 6.4-20240309 revision. Some minor improvements.
• net-snmp: Rebase on XenServer version 5.7.2-52, which incorporates fixes for CVE-2022-24805 and CVE-2022-24809
• openssh: fix CVE-2025-26465
• qemu: Rebuilt with new version of jemalloc.
• qlogic-qla2xxx: Update to version 10.02.12.01_k
• sm: (Storage manager):
  • Logging improvements
  • Minor fixes regarding race conditions
  • Robustify snapshots and a few XAPI calls
  • Send message to XAPI if the garbage collection process doesn't have enough space.
  • Multipath configuration updates for some vendors.
  • Preliminary work for future XOSTOR support and over 2TB VM disks.
• sm-core-libs: fixes.
• vmss: Synchronization with the latest package from XenServer, which replaces the use of a deprecated dependency (imp module) by another.
• xapi:
  • Update to version 24.39.1
  • Many fixes and improvements, among which:
• Improve logging during live storage migration
  • Lengthy VDI migrations were mistakenly canceled upon reaching a 12-hour time limit.
  • Faster starting VMs when they have multiple VIFs or in conditions where the database is under heavy load.
  • High availability occasionally fails to process heartbeats in time when there are a lot of hosts in a pool. Consequently, the host that is unable to process heartbeats is flagged as offline and self-fences.
  • IPv6-related fixes.
  • Configurable threshold for updating last_active
  • Many under the hood improvements or fixes.
  • Added python dependencies for opentelemetry support: pyproject-rpm-macros, python-aiocontextvars, python-charset-normalizer, python-contextvars, python-deprecated, python-idna, python-immutables, python-opentelemetry, python-requests, python-typing-extensions, python-urllib3, python-wheel, python-wrapt, python3-setuptools.
• xcp-ng-release:
  • Sync with xenserver-release-8.4.0-14. (XCP-ng release number remains 8.3.0)
  • Update dependencies between systemd services.
  • Enable new RRDD plugins
• xcp-python-libs: Sync with XenServer, but this only changes packaging details.
• xen: Synchronization with package 4.17.5-6 from XenServer:
  • Fix migration of VMs from XCP-ng 8.2 to XCP-ng 8.3 when the guest is using BHI_DIS_S
  • Initial AMD Turin support
  • Fix dom0 pIRQ limit calculation
  • Fix emulation of BMI1/2 instructions
• xenserver-status-report: Minor update to add scsi disk provisioning mode in the output from this debug tool.
• xo-lite: As described in Xen Orchestra's blog, added VM creation page and form and Display vifs list in vm view and vifs information in side panel
• xs-opam-repo: Update to version 6.86.0 as a dependency for xapi
• xsconsole: Improved xenapi error handling & reintroduced Portable SR feature

Test on XCP-ng 8.3

From an up-to-date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update --enablerepo=xcp-ng-testing
reboot

The usual update rules apply: pool coordinator first, etc.

Versions

• amd-microcode: 20241121-1.1.xcpng8.3
• blktap: 3.55.4-1.1.xcpng8.3
• broadcom-bnxt-en: 1.10.2_223.0.183.0-2.xcpng8.3
• busybox: 1.22.1-7.xcpng8.3
• gpumon: 24.1.0-32.1.xcpng8.3
• guest-templates-json: 2.0.13-1.1.xcpng8.3
• host-upgrade-plugin: 3.0.1-1.xcpng8.3
• intel-i40e: 2.25.11-2.xcpng8.3
• interface-rename: 2.0.6-1.1.xcpng8.3
• ipxe: 20121005-1.0.7.xcpng8.3
• jemalloc: 5.3.0-1.xcpng8.3
• lvm2: 2.02.180-18.1.xcpng8.3
• microsemi-smartpqi: 2.1.30_031-1.xcpng8.3
• ncurses: 6.4-5.20240309.xcpng8.3
• net-snmp: 5.7.2-52.1.xcpng8.3
• openssh: 7.4p1-23.3.2.xcpng8.3
• pyproject-rpm-macros: 1.8.0-4.1.xcpng8.3
• python-aiocontextvars: 0.2.2-3.1.xcpng8.3
• python-charset-normalizer: 2.1.0-4.1.xcpng8.3
• python-contextvars: 2.4-3.1.xcpng8.3
• python-deprecated: 1.2.14-3.1.xcpng8.3
• python-idna: 3.3-4.xcpng8.3
• python-immutables: 0.19-5.xcpng8.3
• python-opentelemetry: 1.12.0-1
• python-requests: 2.28.1-4.1.xcpng8.3
• python-typing-extensions: 3.7.4.3-4.xcpng8.3
• python-urllib3: 1.26.15-4.1.xcpng8.3
• python-wheel: 0.31.1-5.el7_7
• python-wrapt: 1.14.0-4.xcpng8.3
• python3-setuptools: 40.4.1-1.0.1.xcpng8.3
• qemu: 4.2.1-5.2.12.1.xcpng8.3
• qlogic-qla2xxx: 10.02.12.01_k-1.xcpng8.3
• sm: 3.2.12-3.1.xcpng8.3
• sm-core-libs: 1.1.2-1.xcpng8.3
• vmss: 1.2.1-1.xcpng8.3
• xapi: 24.39.1-1.3.xcpng8.3
• xcp-ng-release: 8.3.0-30
• xcp-python-libs: 3.0.4-2.1.xcpng8.3
• xen: 4.17.5-6.1.xcpng8.3
• xenserver-status-report: 2.0.7-1.xcpng8.3
• xo-lite: 0.9.1-1.xcpng8.3
• xs-opam-repo: 6.86.0-1.1.xcpng8.3
• xsconsole: 11.0.8-1.1.xcpng8.3

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

None defined, but early feedback is always better than late feedback, which is in turn better than no feedback

[flakpyro]

@gduperrey

installed on 2 test machines

Machine 1:
Intel Xeon E-2336
SuperMicro board.

Machine 2:
Minisforum MS-01
i9-13900H
32 GB Ram
Using Intel X710 onboard NIC

Both machines installed fine and all VMs came up without issue after. My one test backup job also seemed to run without any issues.

[dthenot]

For people testing the QCOW2 preview, please be informed that you need to update with the QCOW2 repo enabled, if you install the new non QCOW2 version, you risk QCOW2 VDI being dropped from XAPI database until you have installed it and re-scanned the SR.
Dropping from XAPI means losing name-label, description and worse, the links to a VM for these VDI.
There should be a blktap, sm and sm-fairlock update of the same version as above in the QCOW2 repo.

If you have correctly added the QCOW2 repo linked here: https://xcp-ng.org/forum/post/90287

You can update like this:

yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-qcow2
yum update --enablerepo=xcp-ng-testing,xcp-ng-qcow2
reboot

Versions:

• blktap: 3.55.4-1.1.0.qcow2.1.xcpng8.3
• sm: 3.2.12-3.1.0.qcow2.1.xcpng8.3

[ravenet]

@dthenot said in XCP-ng 8.3 updates announcements and testing:

For people testing the QCOW2 preview, please be informed that you need to update with the QCOW2 repo enabled, if you install the new non QCOW2 version, you risk QCOW2 VDI being dropped from XAPI database until you have installed it and re-scanned the SR.
Dropping from XAPI means losing name-label, description and worse, the links to a VM for these VDI.
There should be a blktap, sm and sm-fairlock update of the same version as above in the QCOW2 repo.

If you have correctly added the QCOW2 repo linked here: https://xcp-ng.org/forum/post/90287

You can update like this:

yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-qcow2
yum update --enablerepo=xcp-ng-testing,xcp-ng-qcow2
reboot

Versions:

• blktap: 3.55.4-1.1.0.qcow2.1.xcpng8.3
• sm: 3.2.12-3.1.0.qcow2.1.xcpng8.3

So far so good on Ryzen 1700x host.

[ph7]

Seems to work fine on my old test rig i7-4710MQ with NFS

[gskger]

@gduperrey Updated two Dell R720 servers (dual E5-2640v2, 128GB RAM, with GPUs) and a Dell R730 server (dual E5-2690v4, 512GB RAM, with GPUs) without issues. The services (VMs) are working as expected, so we'll see how the systems perform over the next few days

[Greg_E]

@gskger

Do I need to change each host or just the pool coordinator? Since my lab was recently rebuilt, I might as well get on the testing branch.

[gskger]

@Greg_E You need to update all members of a pool, starting with the pool coordinator (including reboot if needed), followed by each member.

[bufanda]

Upgraded my dev Pool

2 HP EliteDesk 800 G3 Mini
One with i5 6500T and one i7 6700T
Both 32GB RAM, SATA and NVMe Storage
1x Intel NIC
1x RTL8139 NIC

Updated Master first and then the second node. No issues so far.

[Greg_E]

OK,

I have my three hosts in my lab updated, and while the process went smoothly, I'm missing some things.

You can see that the stats for three things are flatline, and that's impossible. This is after a rolling reboot to make sure everything is back in sync.

These hosts are HP T740 thinclients with AMD v1756b processor and x520 network card, and were working properly before going to testing.

The VMs seem to be fine, I only have 2 of them right now. I just migrated one of the VMs to see if anything would change, but stats are still flatline with no data except for RAM which did show the migration.

Stats for each VM seem to show flatline for CPU and Network, but disk is showing some activity (NFS share).

Not sure what else to check at the moment.

[olivierlambert]

Nice catch! I have the same issue here

[stormi]

Indeed, we forgot to tell you about this temporary regression. They're coming back in the next batch of updates, as soon as CI is green!

[olivierlambert]

Hehe congrats @Greg_E for finding the Easter egg then

[Greg_E]

@olivierlambert

I can't do a lot with testing, but sometime I find the odd things that other don't look at. I have been staring at the stats pages for the last few days trying to work out some backup issues, so it was one of those things I just normally looked at.

Also in my lab, with the little tiny processors, I like to keep track of what's going on, on think that lead me to nuke the lab and start over, the old VMs were just not right and might have been down to over provisioned CPU from the old big lab migration.

No surprise, but those stats are also flat line in XO-Lite. I've been using XO-Lite a bit more, I think it's going to be good when you get done building it.

[AlexanderK]

@stormi said in XCP-ng 8.3 updates announcements and testing:

Indeed, we forgot to tell you about this temporary regression. They're coming back in the next batch of updates, as soon as CI is green!

what is CI ?

[gduperrey]

@AlexanderK CI/QA is our testing and integration environment. We run a number of tests to validate that XCP-ng works properly with these new versions, before making them available to our testers in the testing repository.

[abudef]

@Greg_E Please try restarting the toolstack once more.