XCP-ng 8.2 updates announcements and testing

JeffBerntsen

Tested in my test lab and no problems but no Zen CPUs either.

stormi

Thanks for the tests! The update was published: https://xcp-ng.org/blog/2023/09/29/september-2023-security-update/

stormi

New security and maintenance updates

Xen and the linux kernel in the controller domain are updated to fix several vulnerabilities.

We also publish several maintenance updates which were ready and waiting for the next push.

Security updates

xen-*:
- Fix XSA-440 - xenstored: A transaction conflict can crash C Xenstored. XCP-ng uses the ocaml version of xenstored by default, so the issue only concerns users who deliberately switched to C Xenstored.
- Fix XSA-442 - x86/AMD: missing IOMMU TLB flushing. Privilege escalation, DoS, information leaks, on some AMD systems, from VMs with PCI passthrough.
- Fix XSA-443 - Multiple vulnerabilities in libfsimage disk handling. Privilege escalation from PV guests through flaws in libfsimage. Remember that PV guests are not security-supported in XCP-ng 8.2. Despite this, this fix is provided for users who still have PV guests, but we still urge them to convert their VMs to HVM.
- Fix XSA-444 - x86/AMD: Debug Mask handling. DoS provoked from a guest.
kernel:
- Fix XSA-441 - Possible deadlock in Linux kernel event handling. As we understand it, the denial of service would not be possible in XCP-ng's default configuration, but we provide the patched kernel as defence in depth.

Other updates

sm (Storage Manager): improvements around the handling of user customizations on multipath configuration
- Do not overwrite multipath.conf if users made changes
- Add warning to multipath.conf to prevent future modifications (for users which haven't modified it yet, that is, the vast majority)
- Add /etc/multipath/conf.d/custom.conf for user customization
guest-templates-json*: sync with latest Citrix Hypervisor hotfixes. We already had templates for Debian 11, Debian 12, Rocky Linux 9 and CentOS Stream 9 in XCP-ng 8.2. The only new template is Ubuntu 2204.
irqbalance: backport of hotfix XS82ECU1048. "Enable interrupt balancing for Fibre Channel (FC) PCI devices. This improves performance on fast FC HBA SRs, especially if multipathing is used."

Test on XCP-ng 8.2

yum clean metadata --enablerepo=xcp-ng-testing
yum update "xen-*" kernel sm sm-rawhba irqbalance "guest-templates-json*" --enablerepo=xcp-ng-testing
reboot

The usual update rules apply: pool coordinator first, etc.

If you are a user of XOSTOR, change the last parameter to --enablerepo=xcp-ng-testing,xcp-ng-linstor-testing to pick the linstor-compatible version of the sm update.

Versions

xen-*: 4.13.5-9.36.3.xcpng8.2
kernel: 4.19.19-7.0.17.2.xcpng8.2
sm: 2.30.8-2.3.xcpng8.2
irqbalance: 1.0.7-16.xcpng8.2
guest-templates-json*: 1.10.6-1.1.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~2 days due to the security updates.

olivierlambert

Tested on my EPYC host, no issue at all

Danp

Installed yesterday on my R620; no issues experienced thus far.

Andrew

@stormi I have had the updates running for a day on a few machines.

JeffBerntsen

I have it running on my test servers, an HP MicroServer and a ProLiant DL165 G7. No problems so far.

stormi

Thanks for your tests!

Update published: https://xcp-ng.org/blog/2023/10/12/october-2023-security-update/

olivierlambert

\o/ well done, thanks everyone!

stormi

New security update candidates

As promised in the previous security update, here's a new one which goes further, for better defence-in-depth.

Security updates

Xen, XAPI and all its components, sm, libfuse, e2fsprogs: various patches to either deprivilege any component which uses libfsimage, or remove its use altogether. The previous security update fix the urgent flaws described in XSA-443, and this update takes further measures as defence-in-depth.
xs-openssl: rebased on a more recent release from RHEL/CentOS, which fixes various CVEs
kernel: nothing to declare. We just rebuilt it after syncing with XenServer's latest hotfix, but there are no source code changes.

Test on XCP-ng 8.2

yum clean metadata --enablerepo=xcp-ng-testing
yum update --enablerepo=xcp-ng-testing
reboot

The usual update rules apply: pool coordinator first, etc.

If you are a user of XOSTOR, do not install this update candidate. A linstor-compatible version of the sm update is still being prepared.

Versions

blktap: 3.37.4-2.1.xcpng8.2
device-mapper-multipath*: 0.4.9-136.xcpng8.2
e2fsprogs*: 1.47.0-1.1.xcpng8.2
forkexecd: 1.18.3-3.1.xcpng8.2
fuse-libs (added): 2.9.2-10.xcpng8.2
gpumon: 0.18.0-11.1.xcpng8.2
kernel: 4.19.19-7.0.18.1.xcpng8.2
message-switch: 1.23.2-10.1.xcpng8.2
rrd2csv: 1.2.6-8.1.xcpng8.2
rrdd-plugins: 1.10.9-5.1.xcpng8.2
sm: 2.30.8-7.1.xcpng8.2
sm-cli: 0.23.0-54.1.xcpng8.2
sm-rawhba: 2.30.8-7.1.xcpng8.2
squeezed: 0.27.0-11.1.xcpng8.2
varstored-guard: 0.6.2-8.xcpng8.2
vhd-tool: 0.43.0-11.1.xcpng8.2
wsproxy: 1.12.0-12.xcpng8.2
xapi-core: 1.249.32-2.1.xcpng8.2
xapi-nbd: 1.11.0-10.1.xcpng8.2
xapi-storage: 11.19.0_sxm2-10.xcpng8.2
xapi-storage-script: 0.34.1-9.1.xcpng8.2
xapi-tests: 1.249.32-2.1.xcpng8.2
xapi-xe: 1.249.32-2.1.xcpng8.2
xcp-networkd: 0.56.2-8.xcpng8.2
xcp-rrdd: 1.33.2-7.1.xcpng8.2
xen-*: 4.13.5-9.37.1.xcpng8.2
xenopsd*: 0.150.17-2.1.xcpng8.2
xs-openssl-libs: 1.1.1k-9.1.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~3 days

Andrew

@stormi You want us to load ALL of 8.2 testing? There are a few things that show up that were not on your list.... This type of update (and all of the ones in the past) should be in a staging directory so we don't get unintended updates.

stormi

@Andrew Yes. The only other packages in the testing repo are packages that will be pushed to the updates repo at the same time as the rest, but won't be selected by yum as updates for your system.

If you see anything suspicious in the output of yum update, before pressing y, just tell, but I haven't put anything in the testing repo which is not supposed to be published in the next updates.

olivierlambert

Tested here, nothing special to report

JeffBerntsen

Installed on a test system and all is working well for me so far.

stormi

Update published. Thanks for the tests!

https://xcp-ng.org/blog/2023/10/27/october-2023-security-update-2/

JeffBerntsen

@stormi Following the warning in this patch set's announcement, I did not install it on my XOSTOR setup. Is it safe to do so now?

stormi

@JeffBerntsen Yes, we published the XOSTOR-ready version of the update at the same time. The only difference you should notice with other hosts is the installed version of sm (and subpackages) will be 2.30.8-7.1.0.linstor.2.xcpng8.2

JeffBerntsen

@stormi Thanks for the quick reply. That's what I'd figured but I like to be cautious so thought I should ask.

stormi

Urgent security update candidate (only a few hours to provide feedback)

There's an escalation of privilege vulnerability in Intel CPUs of the last few years. It was silently mitigated by Intel in previous microcode updates for the most recent CPU generations, but older affected CPUs were only fixed in yesterday's microcode.

See https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00950.html

Test on XCP-ng 8.2

yum clean metadata --enablerepo=xcp-ng-testing
yum update microcode_ctl --enablerepo=xcp-ng-testing
reboot

The usual update rules apply: pool coordinator first, etc.

Versions

microcode_ctl: 2.1-26.xs26.2.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

If you don't have time for more, just installing the update, rebooting, and checking one VM can start, will be enough.

Test window before official release of the updates

A few hours.

Gheppy

The server (Lenovo System x3650 M5) has started and all 4 VMs are started and functional