XCP-ng 8.2 updates announcements and testing

gduperrey

New security update candidates (xen)

Xen is being updated to mitigate some vulnerabilities:

XSA-326: Malicious guests can cause xenstored to allocate vast amounts of memory, eventually resulting in a Denial of Service (DoS) of xenstored.
XSA-419: Xenstore: Cooperating guests can create arbitrary numbers of nodes
XSA-414: A malicious guest can cause xenstored to crash, resulting in the inability to create new guests or to change the configuration of running guests.
XSA-415: Xenstore: Guests can create orphaned Xenstore nodes
XSA-416: Xenstore: Guests can cause Xenstore to not free temporary memory
XSA-417: Xenstore: Guests can get access to Xenstore nodes of deleted domains
XSA-418: Xenstore: Guests can crash xenstored via exhausting the stack
XSA-420: Oxenstored 32->31 bit integer truncation issues. A malicious or buggy guest can write a packet into the xenstore ring which causes 32-bit builds of oxenstored to busy loop.
XSA-421: Xenstore: Guests can create arbitrary number of nodes via transactions

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing
reboot

Versions:

xen-*: 4.13.4-9.27.1.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~2 days.

Andrew

@gduperrey I upgraded my home/lab machines. One replication backup machine updated. No problems so far but I was not affected by any of the bugs.

olivierlambert

Tested here, seems to work

gskger

@gduperrey Updated my playlab and did some basic tests (create, copy, snapshot, (life-) migrate VMs and disks). Looking good so far.

JeffBerntsen

@gduperrey Tested and working in my lab as well. So far, so good...

gduperrey

The update is published. Thanks for your tests!
Blog post: https://xcp-ng.org/blog/2022/11/04/november-2022-security-update/

gskger

@gduperrey Rolling update of my homelab through Xen Orchestra worked flawlessly. Thanks!

gduperrey

New update candidates (xen, microcode_ctl)

In this release, there are the following fixes and improvements:

xen, microcode_ctl:
- Issues resolved: Minor bug fixes.
- Improvements: Intel microcode is updated to version IPU 2022.3.

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update microcode_ctl xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing
reboot

Versions:
* xen-*: 4.13.4-9.28.1.xcpng8.2
* microcode_ctl: 2:2.1-26.xs23.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

No precise ETA, but the sooner the feedback the better.

olivierlambert

Applied on my EPYC host at home. Nothing specific to report

ravenet

So far fine on an epyc 7002 and a xeon e5 v3

Andrew

@gduperrey Installed on several old and new Intel machines. Working as expected.

gskger

Updated my playlab and nothing to report. Looks good.

stormi

New security update candidate (kernel)

The linux kernel in XCP-ng's domain control is being updated to fix vulnerabilities which may allow a guest to crash to host or make it unresponsive. Even without a malicious attacker, users had reported such issues triggered by the Qlogic/Broadcom netxtreme 2 and the Cisco enic drivers.

It also contains two fixes for issues that were debugged by the XCP-ng developers and the user community, and reported to XenServer developers at the time:

Samba shares failing to reconnect after an unexpected disconnection.
Display issue with Intel NUCs and other hardware, due to a bug in EFI Framebuffer support.

Test on XCP-ng 8.2

From an up to date host:

yum clean metadata --enablerepo=xcp-ng-testing
yum update kernel --enablerepo=xcp-ng-testing
reboot

Versions:

kernel: 4.19.19-7.0.15.1.xcpng8.2

What to test

Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

Test window before official release of the updates

~2 days.

olivierlambert

Tested and working it on my local EPYC box

gskger

Same on my playlab. Updated both hosts and no issues so far.

JeffBerntsen

Both sets of updates installed and tested in my lab with no problems so far.

Andrew

@stormi Running both updates on everything. The 64 bit EFI console on the NUCs works for me with this kernel update.

If you (anyone) is using one of my NUC Test ISO install images then the EFI console will work with the update but the i225/r8125 network may not. To fix that issue, make sure you have installed the network PACKAGE and not just the ISO install. My test ISO installer may not have fully installed the needed package. Download and install the network driver BEFORE the kernel update. If it's too late then you can use a USB stick to just copy the RPM files and install them after the update.

It does not hurt to reinstall the r8125 or the IGC drivers anyway. login to XCP, download driver, install (remove very old driver if there is an error):

wget http://users.ntplx.net/~andrew/xcp/r8125-module-9.009.02-2.xcpng8.2.x86_64.rpm
yum install ./r8125-module-9.009.02-2.xcpng8.2.x86_64.rpm

wget http://users.ntplx.net/~andrew/xcp/igc-module-5.10.146-2.xcpng8.2.x86_64.rpm
yum remove intel-igc-5.10.108-1.xcpng8.2.x86_64
yum install ./igc-module-5.10.146-2.xcpng8.2.x86_64.rpm

Andrew

@stormi I do see this now at boot (related to netdata):

[   49.028835] xenstat.plugin[1818]: segfault at 80 ip 000000000040378a sp 00007ffc4f4278a0 error 4 in xenstat.plugin[400000+8000]
[   49.028842] Code: f4 ff ff 41 b8 68 5d 40 00 b9 d4 00 00 00 ba 30 5f 40 00 be d8 52 40 00 bf 8b 4f 40 00 31 c0 45 31 e4 e8 a9 04 00 00 4c 89 e3 <48> 8b 9b 80 00 00 00 48 85 db 0f 85 be f4 ff ff
e9 b7 f7 ff ff 8b

stormi

@Andrew It never happened before?

stormi

@Andrew said in Updates announcements and testing:

@stormi I do see this now at boot (related to netdata):

[   49.028835] xenstat.plugin[1818]: segfault at 80 ip 000000000040378a sp 00007ffc4f4278a0 error 4 in xenstat.plugin[400000+8000]
[   49.028842] Code: f4 ff ff 41 b8 68 5d 40 00 b9 d4 00 00 00 ba 30 5f 40 00 be d8 52 40 00 bf 8b 4f 40 00 31 c0 45 31 e4 e8 a9 04 00 00 4c 89 e3 <48> 8b 9b 80 00 00 00 48 85 db 0f 85 be f4 ff ff
e9 b7 f7 ff ff 8b

So, I reproduced, but also with the previous kernel, so it's not related to this kernel update.

Update: same regarding the Xen update candidate. Reverting it does not fix the segfault.