XCP-ng 8.2 updates announcements and testing

stormi

Update released (xen + uefistored). Thanks for your tests!

Blog: https://xcp-ng.org/blog/2022/06/13/june-security-update-1/

stormi

New security update (xen, Intel CPUs)

Xen is being updated to mitigate hardware vulnerabilities in Intel CPUs.

Upstream (Xen project) advisory: XSA-404
Citrix Hypervisor Security Bulletin (which also covers vulnerabilities that we already fixed in the previous update): https://support.citrix.com/article/CTX460064/citrix-hypervisor-security-update

Impact of the vulnerabilities - I'll quote Citrix' security team here: "may allow code inside a guest VM to access very small sections of memory data that are actively being used elsewhere on the system"

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing
reboot

Versions:

xen-*: 4.13.4-9.23.1.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~2 days.

JeffBerntsen

@stormi
This seems to be working well on my test pool.

olivierlambert

Same here

gskger

@stormi The initial update worked fine on my playlab (two Dell Optiplex 9010, Intel i5-3550 CPU, Synology shared NFS storage)~~, but I can not migrate VMs between hosts anymore. Neither XO from third party script (commit 395d8, xo-server 5.96.0, xo-web 5.97.2) nor XCP-ng Center 20.04.01 work.~~

Edit: After a complete (bare metal) restart of hosts and storage systems, everything works as expected (create, migrate, start/stop, snapshot of VMs), so I can only blame myself

Andrew

@stormi The update has been OK for me on a bunch of standard machines (older Intel/AMD) over the last day. Normal VM operation, migration, backups, reboots, etc.

I'm having problems with cross host VxLANs, but I can't blame the update for that. I reinitialized the XO SDN plugin and it's working again. It may have something to do with changing pool masters and rebooting the hub server.

stormi

The update is published. Thanks for your tests!

Blog post: https://xcp-ng.org/blog/2022/06/27/june-2022-security-update-2/

NielsH

Thanks for the new update!

We're trying to determine if we are vulnerable to this. In https://xenbits.xen.org/xsa/advisory-404.html I see:

Per Xen's support statement, PCI passthrough should be to trusted

domains because the overall system security depends on factors outside

of Xen's control.

As such, Xen, in a supported configuration, is not vulnerable to

DRPW/SBDR.

Does this mean we are not vulnerable to XSA-404 if we do not use PCI Passthrough?

Cheers,
Niels

stormi

@NielsH DRPW and SBDR are related to MMIO (and thus PCI Passthrough), but there are other vulnerabilities that are not related to it.

NielsH

@stormi said in Updates announcements and testing:

@NielsH DRPW and SBDR are related to MMIO (and thus PCI Passthrough), but there are other vulnerabilities that are not related to it.

Okay, thanks. We have to update it than sadly, heh.
Cheers!

normanghenderson

Hi, I've just applied the May and two June security updates (total of 8 patches) to a test box, no problems. But before I try these on the live pool: can anyone tell me if a reboot is required? The blog posts referenced do not mention this.

stormi

@normanghenderson They do

normanghenderson

@stormi Doh. How did I miss that?

stormi

New maintenance update candidate (xs-openssl, xcp-ng-xapi-plugins, blktap, sm)

Several package updates that we had queued for a future update are ready for you to test them.

xs-openssl:
- was rebuilt without compression support. Although compression was not offered by default and the clients that connect to port 443 of XCP-ng hosts don't enable compression by default, it's better security-wise not to support it at all (due to CRIME), and this will make security scanners happier.
- received a patch from RHEL 8's openssl which fixes a potential denial of service: "CVE-2022-0778 openssl: Infinite loop in BN_mod_sqrt() reachable when parsing certificates"
xcp-ng-xapi-plugins received a few fixes:
- Avoid accidentally installing updates from repositories that users may have enabled on XCP-ng (even if they should never do this), when using the updater plugin (Xen Orchestra uses it to install updates).
- In the updater plugin again, error handling was broken: whenever an error would occur (such as a network issue preventing from installing the updates), another error would be raised from the error handler, and thus mask the actual reason for the initial error. That's what happens when you write command with 3 m .
blktap: received a fix backported from one if Citrix Hypervisor's hotfixes, which addresses a possible segmentation fault if you create a lot of snapshots at the same time.
sm ("Storage Manager", responsible for the SMAPIv1 storage management layer) received a few fixes:
- We fixed an issue with local ISO SRs and mountpoints: creating a local ISO SR on a directory that is a mountpoint for another filesystem would unmount it. The patch was not accepted upstream because it touches legacy code that Citrix won't support, according to the developer who answered, but we considered it safe and useful enough to apply it to XCP-ng anyway.
- The (experimental) MooseFS driver will now default to creating a subdirectory in the mounted directory, to avoid collision between several SRs using the same share.
- The update also includes the followings fix from one of Citrix Hypervisor's hotfixes: CA-352880: when deleting an HBA SR remove the kernel devices
- Two other fixes which are hard to explain in user terms but typically don't affect the majority of users.

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update blktap sm sm-rawhba xcp-ng-xapi-plugins xs-openssl-libs --enablerepo=xcp-ng-testing
xe-toolstack-restart

Toolstack restart should be enough but I haven't checked yet that it does restart the TLS tunnel, so a reboot may be safer to ensure you actually test the updated packages.

Versions:

xs-openssl-libs: 1.1.1k-6.1.xcpng8.2
xcp-ng-xapi-plugins: 1.7.2-1.xcpng8.2
blktap: 3.37.4-1.0.1.xcpng8.2
sm, sm-rawhba: ~~2.30.7-1.2.xcpng8.2~~ updated 2022-07-08 to fix a regression: 2.30.7-1.3.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

No precise ETA, but the sooner the feedback the better.

olivierlambert

Tested with success here

gskger

@stormi All good with updating my playlab.

JeffBerntsen

Seems to be working fine for me as well.

stormi

We found and fixed a regression in the update candidate for sm regarding shared thick-provisioned drivers based on LVM (lvmoiscsi and others).

Please update again (mirrors will receive the within 15 minutes) using the command given above. This will give you sm and sm-rawhba in version-release 2.30.7-1.3.xcpng8.2.

gskger

@stormi Can not help with the sm and sm-rawhba regression, but did the update anyway. Worked well and kicking some VMs around works as expected. Let's see how my playlab does during the weekend.

AlexD2006

@stormi
is there some information about that regression?
maybe i can help to test.
have a lab pool here with 3 hosts on a iscsi SR.