XCP-ng 8.2 updates announcements and testing

JeffBerntsen

This is installed and working on my two test systems but they're both AMD so I'm not able to test the updated microcode.

Andrew

@bleader Updates running on several old and new intel machines (including microcode update). Working fine so far. Rolling Pool Reboot is a helpful feature.

bleader

The update has been published, thank you for testing it out.

https://xcp-ng.org/blog/2024/03/15/march-2024-security-update/

gduperrey

New update candidates for you to test!

As you may know, we group non-urgent updates together for a collective release, in order not to cause unnecessary maintenance for our users.

The moment to release such a batch has come, so here they are, ready for user tests before the final release.

openvswitch:
- CVE-2023-1668: Correct a flaw when processing an IP packet with protocol 0.
- CVE-2023-5366: Apply the patch for OpenFlow and neightbor discovery target with IPv6
- CVE-2023-3966: Correct a vulnerabity with "crafted Geneve packets causing invalid memory accesses and potential denial of service".
blktap:
- Synced with XS82ECU1056:
  - Bugfix for time out on NFS tasks which can sometimes exceed the configured value.
  - Improve the error handling for some lost iSCSI connection.
sm:
- Support NFS servers which only offer NFSv4. The discovery process for such servers differs from that of servers which offer also NFSv3, so the SR driver had to be improved.
- Synced with XS82ECU1056: bugfix on the path checker for DELL EqualLogic with iSCSI protocol
- Synced with XS82ECU1060: bugfix for when a host is unable to log into all iSCSI portals because there are separate independent Target Portal Groups inside the IQN.
util-linux: preparatory steps to support 4k-only disks.
xapi: Bugfix in a testing framework.
xcp-ng-pv-tools: Small fixes regarding VM stats reporting.
xcp-ng-xapi-plugins: Add check_installed function in updater plugin to test installed packages. This is a prerequisite for the upcoming XOSTOR release.

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update --enablerepo=xcp-ng-testing blktap openvswitch sm-* util-linux xapi-* xcp-ng-pv-tools xcp-ng-xapi-plugins
reboot

The usual update rules apply: pool coordinator first, etc.

Versions

blktap: 3.37.4-3.1.xcpng8.2
openvswitch: 2.5.3-2.3.12.2.xcpng8.2
sm: 2.30.8-10.1.xcpng8.2
util-linux: 2.23.2-52.1.xcpng8.2
xapi: 1.249.32-2.2.xcpng8.2
xcp-ng-pv-tools: 8.2.0-12.xcpng8.2
xcp-ng-xapi-plugins: 1.10.0-1.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~1 week.

Andrew

@gduperrey Updates installed and running.

JeffBerntsen

They're running without problems for me on my test systems

olivierlambert

Tested and working here

JamesG

@gduperrey Applicable to 8.3?

gskger

@gduperrey Succesfully updated my two host pool. Let's see how the weekend goes with some tests.

stormi

@JamesG No, it's just for XCP-ng 8.2

gduperrey

The updates have been published; thank you for testing them out.

https://xcp-ng.org/blog/2024/03/29/march-2024-maintenance-update/

bleader

New security update candidate (xen)

Three new XSAs were published on 9th of April.

Notes:

XSA-456 was published on various public mailing list but its entry is not yet on the xenbits page, hence the different link for this one.
XSAs description to be completed later, early posting to provide more chances to run tests before final release.

XSA-454 impacts all host running HVM or PVH guests on x86_64, therefore all supported architectures on XCP-ng.
XSA-455 relates to XSA-407 (Branch Type Confusion) having a logical error, check its VULNERABLE SYSTEMS section for impacted systems.
XSA-456 should only impact Intel CPU as it is understood at this time.

SECURITY UPDATES

xen-*:
- Fix XSA-454 - x86 HVM hypercalls may trigger Xen bug check. HVM and PVH guests can DoS a host in some cases calling 32-bit-mode hypercalls with parameters that will lead the hypercall sanity checks to trigger a crash.
- Fix XSA-455 - x86: Incorrect logic for BTC/SRSO mitigations. Fix for XSA-407 was not properly used, meaning an attacker could be able to infer memory from host or other guests. All versions since 4.13.4-9.24.1 are vulnerable.
- Fix XSA-456 - x86: Native Branch History Injection. An attacker could infer memory of host or other guests by using the Native Branch History Ijnection flaw. This is an evolution of Spectre-BHB which was previously considered not to be a risk for Xen.

Test on XCP-ng 8.2

yum clean metadata --enablerepo=xcp-ng-testing
yum update "xen-*" --enablerepo=xcp-ng-testing
reboot

The usual update rules apply: pool coordinator first, etc.

Versions:

xen: 4.13.5-9.40.1.xcpng8.2

What to test

Normal use and anything else you want to test.

Test window before official release of the updates

~1 days because of security updates.

Andrew

@bleader Did they get published to the right directory? I don't see anything in testing (stuff is in incoming).

bleader

My bad, we were a bit late and I tried to be quick and forgot to move it... Just did that, should be good soon, it needs some time to sync repos.

olivierlambert

Tested in my home lab, no explosion

gskger

@bleader Updated my homelab without any issues

JeffBerntsen

@bleader Seems to be running just fine on my test servers as well.

Andrew

@bleader Installed and running.

stormi

Now live. Thanks everyone!

https://xcp-ng.org/blog/2024/04/13/april-2024-security-update/

stormi

New update candidates for you to test!

As you may know, we group non-urgent updates together for a collective release, in order not to cause unnecessary maintenance for our users.

The moment to release such a batch has come, grouped with a recent security fix, so here they are, ready for user tests before the final release.

qemu: security fix to protect hosts from DoS that can be caused by a malicious administrator of a guest (XenServer security bulletin)
openssh: rebased on CentOS 7's openssh-7.4p1-23.el7_9 to fix various CVEs. The update also changes the way default ciphers and algorithms are set. See dedicated section below.
curl: updated to version 8.6.0 + patches, to fix several CVEs. Note: in XCP-ng, curl is mostly used by yum for downloading RPMs when updating hosts.
sudo: updated to a recent release to fix some CVEs (none critical in the context of XCP-ng as far as we can tell)
Note : XenServer published updates for openssh, curl and sudo together as hotfix XS82ECU1063, whose only description is "This hotfix includes upstream code changes that may reduce false-positive reports for the following CVEs: CVE-2023-38545, CVE-2023-48795 and CVE-2023-28486." We are not sure what this "false-positive reports" statement means, but what appears to us is that anyway the CVEs fixed were clearly either not exploitable, or not critical, in the context of XCP-ng.
microcode_ctl: updated to Intel's IPU 2024.2 release, + a fix to Gemini Lake 06-7a-01 for a regression that was introduced by IPU 2024.1 (see this forum thread and this one too)
linux-firmware: updated AMD firmware to the 2024-05-03 drop. What fixes this contains exactly is not described.
XAPI and related components: synced with Citrix Hypervisor hotfixes XS82ECU1064 and XS82ECU1053. Various fixes. Check the hotfixes descriptions.
- We also added a fix to make the small web server managed by XAPI report accurate mimetypes for files it serves. This is important for XO Lite (which is not installed by default on XCP-ng 8.2, but can be if you need it).
tzdata: updated timezone data.
sm: adds the new largeblock storage driver, which is a local SR driver which workarounds the current limitation our storage stack has with 4KiB-block-only devices, by transparently emulating a 512B block size (at some performance cost, obviously). More about it in this forum thread.
- Also rebased on Citrix Hypervisor's hotfix XS82ECU1065

About OpenSSH, Ciphers and algorithms

To ensure that the lists of authorized Ciphers, algorithms, etc., defined by XenServer's security team are applied, XenServer packagers had decided that any change they had to make would plainly overwrite /etc/ssh/sshd_config and /etc/ssh/ssh_config. Although we discourage customizing XCP-ng's configuration too far, we didn't think it would be acceptable for our users than these files be overwritten without any notice.

So we looked for another approach, and decided for this: we don't define these keys (Ciphers, MACs, KexAlgorithms, HostKeyAlgorithms) in the configuration files anymore. Now, we define them at build time, directly in the built binaries.

If you need to override them, you can still do so in the configuration files. But then this means you become responsible of their future update, whenever a cipher or algorithm starts being considered weak, as this will override the built-in settings defined by our security team.

The update process will attempt to be smart and will remove the definition of the above keys from /etc/ssh/sshd_config and /etc/ssh/ssh_config, but only if you had not touched these lines. If you have brought customizations to these keys, then we will leave them as they were. In this case, this means that any future change our security team may make to the built-in values will not be applied to your hosts, because your changes in the configuration files will override the built-ins.. If you are in this situation, you have to choose: either remove these lines manually, or make sure you keep them updated by yourself according to your security policy.

You can check what configuration is applied to your instance of sshd with: sshd -T.

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-candidates
yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates
reboot

The usual update rules apply: pool coordinator first, etc.

Versions

List based on source RPMs, which can differ from actual built RPMs (for example, the xapi source RPM, when built, produces xapi-core, xapi-xe and xapi-tests).

curl-8.6.0-2.1.xcpng8.2
forkexecd-1.18.3-10.1.xcpng8.2
gpumon-0.18.0-18.1.xcpng8.2
linux-firmware-20190314-11.1.xcpng8.2
message-switch-1.23.2-17.1.xcpng8.2
microcode_ctl-2.1-26.xs29.2.xcpng8.2
microsemi-smartpqi-alt-2.1.28_025-1.xcpng8.2
ocaml-rrd-transport-1.16.1-15.1.xcpng8.2
ocaml-rrdd-plugin-1.9.1-15.1.xcpng8.2
ocaml-tapctl-1.5.1-15.1.xcpng8.2
ocaml-xcp-idl-1.96.7-4.1.xcpng8.2
ocaml-xen-api-client-1.9.0-18.1.xcpng8.2
ocaml-xen-api-libs-transitional-2.25.6-7.1.xcpng8.2
openssh-7.4p1-23.2.1.xcpng8.2
qemu-4.2.1-4.6.4.1.xcpng8.2
rrd2csv-1.2.6-15.1.xcpng8.2
rrdd-plugins-1.10.9-12.1.xcpng8.2
sm-2.30.8-12.2.xcpng8.2
sm-cli-0.23.0-61.1.xcpng8.2
squeezed-0.27.0-18.1.xcpng8.2
sudo-1.9.15-2.1.xcpng8.2
tzdata-2024a-1.el7
varstored-guard-0.6.2-15.xcpng8.2
vhd-tool-0.43.0-18.1.xcpng8.2
wsproxy-1.12.0-19.xcpng8.2
xapi-1.249.36-1.1.xcpng8.2
xapi-nbd-1.11.0-17.1.xcpng8.2
xapi-storage-11.19.0_sxm2-17.xcpng8.2
xapi-storage-script-0.34.1-16.1.xcpng8.2
xcp-networkd-0.56.2-15.xcpng8.2
xcp-ng-release-8.2.1-11
xcp-rrdd-1.33.4-4.1.xcpng8.2
xenopsd-0.150.19-3.1.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Extra attention to:

SSH server, ciphers, configuration files
checking downloads with curl
sudo if you use it

Test window before official release of the updates

Release planned either on Friday 14th or Monday 17th.