XCP-ng 8.2 updates announcements and testing

ravenet

@gduperrey Seems fine here as well under basic usage.

theAeon

@gduperrey Skylake-S Xeon here w/o patched bios, works fine so far.

edit: double checked /proc/cpuinfo, seems the ucode update is applying correctly, so, yeah, it worked.

olivierlambert

Tested on my EPYC test bench (so not really relevant), but at least nothing broke

gskger

@gduperrey Updated my two host playlab (Dell Optiplex 9010, Intel i5-3550 CPU). Everything works as expected, but I doubt my CPUs are relevant for the update either.

I have a strange INTERNAL_ERROR((Failure "Expected string, got 'N'")) error with Xen Orchestra (from 3rd party script update to commit a1bcd) when creating a new Debian 11 VM as part of my test procedure, but I could continue with XCP-ng Center. Just found and will follow xoce INTERNAL_ERROR while trying to create VM.

olivierlambert

Yes, it's likely unrelated Thanks for the report @gskger !

gduperrey

Update released. Thanks everyone for testing!

https://xcp-ng.org/blog/2022/05/16/may-2022-security-update/

stormi

New update candidate: uefistored

As microsoft.com recently blocked the user agent our secureboot-certs script uses to download UEFI Secure Boot certificates from them, we took the following actions:

Documented how to download and install the certificates manually: https://xcp-ng.org/docs/guides.html#install-the-default-uefi-certificates-manually
Changed the user agent in secureboot-certs to make the automated download and installation possible again.
Added a new --user-agent parameter to secureboot-certs install to let you override the default easily in case of future need.
Improved the error message in case of download failure to 1. let users know about the --user-agent parameter and 2. provide the link towards the manual installation instructions.

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update uefistored --enablerepo=xcp-ng-testing

No toolstack restart or reboot needed.

Versions:

uefistored: 1.1.5-1.xcpng8.2.x86_64

What to test

UEFI VMs. Secure Boot. Installation of certificates using secureboot-certs install: manual install, automated install with default user agent, automated install with --user-agent parameter.

Test window before official release of the updates

~ 1 week. Maybe more if it allows to synchronise with other updates not too far in the future.

Andrew

@stormi It installs and runs....

The "help" does not mention the user-agent option.

stormi

@Andrew That's because install is a sub-command: secureboot-certs install -h.

Anyway, if download fails (you can test by using "test" as the user agent for example), the option will be mentioned.

stormi

To me, this uefistored update is ready, but I'll group it with the next updates.

Test feedback remains welcome.

stormi

New security update (xen)

Impact: when the conditions are met (roughly: CPU Model, PV guest + PCI passthrough or race condition exploitation), an attacker in a malicious VM may escalate privilege and control the whole host.

Upstream (Xen project) references: XSA-401 and XSA-402

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing
reboot

Versions:

xen-*: 4.13.4-9.22.2.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~2 days.

gskger

@stormi Update worked fine and no problems so far. Did the usual tests to create, move, snapshot, backup and restored some Linux and Windows VMs.

Andrew

@stormi I've had it running for 24 hours on several active machines doing the usual jobs. Seems good.

olivierlambert

Same here

stormi

Update released (xen + uefistored). Thanks for your tests!

Blog: https://xcp-ng.org/blog/2022/06/13/june-security-update-1/

stormi

New security update (xen, Intel CPUs)

Xen is being updated to mitigate hardware vulnerabilities in Intel CPUs.

Upstream (Xen project) advisory: XSA-404
Citrix Hypervisor Security Bulletin (which also covers vulnerabilities that we already fixed in the previous update): https://support.citrix.com/article/CTX460064/citrix-hypervisor-security-update

Impact of the vulnerabilities - I'll quote Citrix' security team here: "may allow code inside a guest VM to access very small sections of memory data that are actively being used elsewhere on the system"

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing
reboot

Versions:

xen-*: 4.13.4-9.23.1.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~2 days.

JeffBerntsen

@stormi
This seems to be working well on my test pool.

olivierlambert

Same here

gskger

@stormi The initial update worked fine on my playlab (two Dell Optiplex 9010, Intel i5-3550 CPU, Synology shared NFS storage)~~, but I can not migrate VMs between hosts anymore. Neither XO from third party script (commit 395d8, xo-server 5.96.0, xo-web 5.97.2) nor XCP-ng Center 20.04.01 work.~~

Edit: After a complete (bare metal) restart of hosts and storage systems, everything works as expected (create, migrate, start/stop, snapshot of VMs), so I can only blame myself

Andrew

@stormi The update has been OK for me on a bunch of standard machines (older Intel/AMD) over the last day. Normal VM operation, migration, backups, reboots, etc.

I'm having problems with cross host VxLANs, but I can't blame the update for that. I reinitialized the XO SDN plugin and it's working again. It may have something to do with changing pool masters and rebooting the hub server.