XCP-ng 8.3 updates announcements and testing

stormi

New security update candidates for XCP-ng 8.3 LTS (xen, intel-microcode)

Two new XSAs were published on November 12th 2024.
Intel published a microcode update on the November 12th 2024.

XSA-463 an unprivileged guest making two quick accesses to the VGA memory can deadlock a host.
XSA-464 an unprivileged PVH guest may access sensitive information from the host, control domain or other guests.

SECURITY UPDATES

xen-*:
- Fix XSA-463 - Deadlock in x86 HVM standard VGA handling. A mistake in the locking of process of the "standard" VGA memory makes it possible for a guest to make 2 quick accesses and create a deadlock that will hang the host.
- Fix XSA-464 - libxl leaks data to PVH guests via ACPI tables. The ACPI tables for PVH guests initialization left the excess memory space with its previous content, which was then copied to the guest memory as it was, resulting in possible leak of sensitive information. This doesn't affect XCP-ng in its normal configuration, as only HVM and PV-in-PVH (not affected) guests are supported.
intel-microcode:
- Latest Intel microcode update, published on November the 12th:
  - Security updates for INTEL-SA-01101
  - Security updates for INTEL-SA-01079
  - Updated security updates for INTEL-SA-01097
  - Updated security updates for INTEL-SA-01103
  - Multiple other updates for functional issues.

Other updates

XO Lite: updated to version 0.5.0, fixing its loading without internet access and bringing some other improvements. Changelog: https://github.com/vatesfr/xen-orchestra/blob/xo-lite-v0.5.0/%40xen-orchestra/lite/CHANGELOG.md

Test on XCP-ng 8.3

yum clean metadata --enablerepo=xcp-ng-candidates
yum update  --enablerepo=xcp-ng-candidates
reboot

The usual update rules apply: pool coordinator first, etc.

Versions:

Security updates:

xen: 4.17.5-4.xcpng8.3
intel-microcode: 20241016-1.xcpng8.3

Maintenance update:

xo-lite: 0.5.0-1.xcpng8.3

What to test

Normal use and anything else you want to test.

Test window before official release of the update

~ 2 day because of security updates.

flakpyro

@stormi Updated a test machine running only couple VMs. Everything installed fine and rebooted without issue.

Machine is:
Intel Xeon E-2336
SuperMicro board.
One VM happens to be windows based with an Nvidia GPU passed though to it running Blue Iris using the MSR fixed found elsewhere on these forums, fix continues to work with this version of Xen.

Andrew

@stormi Installed on several test and pre-production machines.

XCP-ng-JustGreat

Latest version 8.3 candidate updates installed and are working fine on three-host home lab pool. Received a couple of repo errors for a certain mirror, but yum tried another mirror and it completed successfully. After updates were applied, performed live migrations between hosts with no problems and updated a Windows 11 Version 24H2 VM to the November 2024 cumulative update without problems. (VM is currently running Citrix Tools 9.3.2 without issues.)

gduperrey

Update published: https://xcp-ng.org/blog/2024/11/15/november-2024-security-update-for-xcp-ng-8-3/

Thank you for the tests!

flakpyro

@gduperrey Installed on a 2 host AMD based test pool, as well as our 5 host Intel based production pool without issue using rolling pool update. Everything migrated, updated then migrated back perfectly.

Also installed on my home server without issue.

Greg_E

Has the kernel changed to a newer version, or still 4.xx.xx?

gduperrey

@Greg_E No, the kernel version has not changed.
There is no kernel update in this update series either.

dxym

The blog(https://xcp-ng.org/blog/2024/11/15/november-2024-security-update-for-xcp-ng-8-3/) states the following:

Host reboots are necessary after this update.

However, the command output indicates:

# needs-restarting -r
No core libraries or services have been updated.
Reboot is probably not necessary.

Which one is correct?
It might be better to reboot the host, but not everyone checks the blog regularly.

gduperrey

@dxym It is always important to follow the instructions given on the forum or on the blog. In both cases, we indicate that the hosts must be restarted.
This way, we are sure that the hosts will apply the changes coming from the updates, like here changes on Xen and the Intel microcode.

stormi

needs-restarting is a tool from CentOS, which is not aware of the reality of XCP-ng. It's not even able to detect that a Xen or a microcode update requires a reboot. So, as Gaël says.

gduperrey

New update candidates for you to test!

A new batch of non-urgent updates is ready for user tests before a future collective release. Below are the details about these.

amd-microcode: Update AMD microcode to the 2024-11-21 drop
- Updates firmware for families 17h and 19h CPUs. For the first time, AMD published updates for non-server CPUs. One can assume that they started supporting microcode update for these, contrarily to what they did in the past, and that these updates thus fix various bugs and vulnerabilities. This is only (sensible) speculation at the moment, though.
grub: Backport VLAN networking support for UEFI PXE boot.
iperf3: Upgrade to version 3.9-13 from CentOS 7
- Includes a security fix for CVE-2023-38403
kernel: Backport of a fix to correct cooling fan rotation speed on some Lenovo servers. For more information, you can read this thread on the forum.
kexec-tools: Backport of a patch removing kernel_version(). Fixing a bug for kernel with a patchlevel greater than 255.
netdata: Fixed an issue that could occur when quickly uninstalling the package, right after an unfinished installation, and leave a service in an undetermined status.
slang: Fixed display and input issues in optional package mc.
sm: Contains a fix for leaf coalesce where the size of the leaf to coalesce was wrongly computed before deciding if it was a live coalesce or not, it resulted in some leaf having too much data to coalesce not successing the live coalesce and staying in this state indefinitely.
xapi:
- Fixed a malfunction related to the absence of a certificate, which could cause a loop.
- Fixed and improved various points in IPv6, related to management, reboot and re-initialization.
xo-lite: Update to version 0.6.0. For more details, you can consult the blog post on the latest release of Xen Orchestra.

Optional packages:

kernel-alt: Backport of a fix to correct cooling fan rotation speed on some Lenovo servers. For more information, you can read this thread on the forum.
socat: Update the package to version 1.7.4.1 which includes a fix for a buffer overflow and security fixes.
traceroute: Updated to version 2.1.5.
Alternate Drivers: Updated to newer versions.
- broadcom-bnxt-en-alt: From version 1.10.2_227.0.130.0 to 1.10.3_231.0.162.0
- intel-i40e-alt: From version 2.22.20-3.1 to 2.26.8
- More information about drivers and current versions is on the drivers page: (https://github.com/xcp-ng/xcp/wiki/Drivers).

Test on XCP-ng 8.3

From an up-to-date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update --enablerepo=xcp-ng-testing
reboot

The usual update rules apply: pool coordinator first, etc.

Versions

amd-microcode: 20240503-1.1.xcpng8.3
grub: 2.06-4.0.2.1.xcpng8.3
iperf3: 3.9-13.xcpng8.3
kernel: 4.19.19-8.0.37.1.xcpng8.3
kexec-tools: 2.0.15-20.1.xcpng8.3
netdata: 1.44.3-1.2.xcpng8.3
slang: 2.3.2-11.xcpng8.3
sm: 3.2.3-1.13.xcpng8.3
xapi: 24.19.2-1.9.xcpng8.3
xo-lite: 0.6.0-1.xcpng8.3

Optional packages:

kernel-alt: 4.19.322+1-1.xcpng8.3
socat: 1.7.4.1-6.xcpng8.3
traceroute: 2.1.5-2.xcpng8.3
Alternate drivers:
- broadcom-bnxt-en-alt: 1.10.3_231.0.162.0-1.xcpng8.3
- intel-i40e-alt: 2.26.8-1.xcpng8.3

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

None defined, but early feedback is always better than late feedback, which is in turn better than no feedback

flakpyro

@gduperrey installed on 2 test machines

Machine 1:
Intel Xeon E-2336
SuperMicro board.

Machine 2:
Minisforum MS-01
i9-13900H
32 GB Ram
Using Intel X710 onboard NIC

Both machines installed fine and all VMs came up without issue after.

I ran a backup job after to test snapshot coalesce, no issues there.

ravenet

@gduperrey

Tested on Multiple systems. Ryzen 1700x andThreadripper 5975. fine so far

Andrew

@gduperrey I have several hosts updated and running. I'm happy to see 8.3 updates on parity with 8.2.

ravenet

@gduperrey
Tested on 4 systems in production

Ryzen 1700x, on asrock rack mb w radeon pro GPU pass through
Threadripper 5975wx on asrock rack mb w radeon pro GPU pass through
Epyc 9224 on Asus
Epyc 7313P on Asus

XCP-ng-JustGreat

Applied latest candidate test updates to 3 x Dell OptiPlex 7040 (i7-6700, 48GB, 10Gbps-attached TrueNAS shared-storage) pool. Update process was error-free and successful. Everything appears to be working normally following the update.

gskger

@gduperrey Update some Dell R720s with GPUs and a Dell R730. Update worked without any problem and VMs operate as expected. Will update this post if that changes during day-to-day operation. Great work!

gduperrey

New update candidates for you to test!

In addition to the previous updates, available for testing, here are some new, non-urgent ones, expected to be released in a few days. Below are the details about these.

blktap: Add an option to use backup footer when vhd-util query is called. This will be used in a future storage driver.
intel-igc: Update to version 5.10.226.
xcp-ng-deps: Added vim-minimal as a dependency, so it is always present on XCP-ng systems.

For XOSTOR users:

drbd: Prevent a dead-lock in some situations, plus other improvements

(Reminder: XOSTOR is still in beta stage on XCP-ng 8.3)

Test on XCP-ng 8.3

From an up-to-date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update --enablerepo=xcp-ng-testing
reboot

The usual update rules apply: pool coordinator first, etc.

Versions

blktap: 3.54.9-1.2.xcpng8.3
intel-igc: 5.10.226-1.xcpng8.3
kmod-drbd: 9.2.11-1.1.xcpng8.3
xcp-ng-deps: 8.3-13

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~ 2 days

flakpyro

@gduperrey Installed on the same 2 hosts as the last batch of test updates released in December.

No issues to report so far, ran a backup job after after without issue.