XCP-ng 8.2 updates announcements and testing

stormi

I have pushed an updated bnxt_en driver as broadcom-bnxt-en-alt for 8.0 and 8.1. It is in xcp-ng-testing for 8.0. If fixes a very specific issue from what I can see in the code diff, so no need to install it if everything is already OK for you.

stormi

By the way, is anyone interested (and why) in updated drivers for cisco-enic and cisco-fnic?

onyxfire

This post is deleted!

stormi

This post is deleted!

stormi

First update for XCP-ng 8.1:

xcp-ng-deps for pulling the gpumon package. That issue has kept us busy this week when people started reporting installed or upgraded systems that would be unreachable from XO and XCP-ng Center after a reboot. Turns out that XAPI needs gpumon, a package that can only be built using a proprietary (from what I remember. One would need to check the license) nVIDIA developement toolkit. I had removed it from XCP-ng 8.1 because I was convinced that it was only necessary with the vgpu feature with nVIDIA, and we don't have the vgpu package in XCP-ng because it is closed-source. XAPI's start process stalls without an error message if there's an nVIDIA GPU. Installing gpumon fixes it.
- after the update: if you were affected by the issue (you would know), you may need to do an emergency network restart and possibly reconnect the SRs. There may be other consequences for the hosts, such as missing removable media from VMs and possibly others. If you have a way to come back to a backup and reinstall with the fixed ISO (released 2020-04-06), this may be the safest path.
- see also https://github.com/xcp-ng/xcp/wiki/XCP-ng-8.1-Known-Issues#host-unreachable-after-upgrade-or-fresh-installation-on-hosts-having-an-nvidia-gpu
xcp-ng-release* for reducing chrony-wait's timeout from 600s to 120s. So if your host can't connect the ntp server that was configured at installation, you'll only have to wait for 2 minutes, not 10. But your hosts really should be able to connect a ntp server. Accurate time is required to avoid timing and synchronisation issues.

No reboot required if your host is already behaving correctly. If you have a discrete nVIDIA GPU and the host had no issue, then 1. I'm surprised, 2. I advise to reboot.

New installation ISOs including those two updates will be released shortly. 2020-04-06 update: they have been released, named xcp-ng-8.1.0-2.iso and xcp-ng-8.1.0-2-netinstall.iso.

As ususal, see https://github.com/xcp-ng/xcp/wiki/Updates-Howto

marekm

@stormi Is this related to https://bugs.xenserver.org/browse/XSO-936 (build dependency on gdk-devel) - or a different issue? (I suspect gdk-devel is some part of GNOME, not nVidia proprietary - but could be wrong.)

stormi

@marekm said in Updates announcements and testing:

@stormi Is this related to https://bugs.xenserver.org/browse/XSO-936 (build dependency on gdk-devel) - or a different issue? (I suspect gdk-devel is some part of GNOME, not nVidia proprietary - but could be wrong.)

Yes it is this issue, and no, it's not GNOME's gdk-devel. Here GDK stands for GPU Deployment Kit. It's unfortunate that someone at some point thought that it would be a good idea to name the dependency gdk-devel, which was already taken by the GNOME project.

stormi

By the way, if someone wants to have a look at how to build gpumon from its source RPM and prove me wrong about the impossibility to do it only with FOSS tools, I'll gladly revise my judgment

marekm

@stormi OK, just asked since the issue is about a year old and still unanswered by Citrix, wondering why it didn't show up in earlier xcp-ng releases. Perhaps they have only recently made some change to XAPI so it requires gpumon, and this change could be reverted (or better yet, fixed to handle the missing package gracefully, not hang the whole machine at boot time).

stormi

@marekm said in Updates announcements and testing:

@stormi OK, just asked since the issue is about a year old and still unanswered by Citrix, wondering why it didn't show up in earlier xcp-ng releases. Perhaps they have only recently made some change to XAPI so it requires gpumon, and this change could be reverted (or better yet, fixed to handle the missing package gracefully, not hang the whole machine at boot time).

It's simpler: it's only with XCP-ng 8.1 that I have removed gpumon. That was a mistake so I had to put it back.

marekm

@stormi Thanks. Is this https://github.com/xenserver/gpumon - if yes, it doesn't look like something big, perhaps could be patched to remove dependency on proprietary nvidia stuff.

stormi

Yes, it's this. I've also create an issue for the XAPI project, that got good reception from the devs. All agree that XAPI should be able to start even if gpumon is not there.

stormi

Note that patching gpumon so that it doesn't depend on nVIDIA stuff probably just means making it a stub service that always answers the same regardless of the actual state of the GPUs.

stormi

Following yesterday's Xen security updates released by Citrix, here are test packages with security patches. As usual with security matters, they need to be tested quickly so that we can release them to everyone fast.

On XCP-ng 8.1

# on an up to date host
yum clean all --enablerepo=xcp-ng-testing
yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing

reboot (master first)

On XCP-ng 8.0

This batch of updates contains not only security fixes but also bug fixes that I had queued for the next patch train.

# on an up to date host
yum clean all --enablerepo=xcp-ng-testing
yum update bugtool-conn-tests qlogic-netxtreme2 qlogic-netxtreme2-4.19.0+1-modules sm sm-rawhba xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools xenserver-status-report --enablerepo=xcp-ng-testing

reboot (master first)

On XCP-ng 7.6

XCP-ng 7.6 is not supported anymore, so there are no updates available for it.

stormi

Hi users!

Today's the last day to install the update candidate and give feedback. If you have any test host, please install and reboot. That's it and it's useful to us.

stormi

Update released: April 2020 XCP-ng Security Updates

stormi

Intel CPUs "CROSSTalk" vulnerability.

Following the disclosure of the CROSSTalk CPU vulnerabilities and the release of updated microcode by Intel, here are update candidates for XCP-ng 8.0 and 8.1. Prompt feedback by all available testers is wanted.

Details and discussion in the dedicated thread.

stormi

Update published: https://xcp-ng.org/blog/2020/06/12/intel-microcode-security-update-crosstalk/

stormi

I haven't had many tests from the community for the previous updates.

I'm still trying to convince myself that I can count on the community to test updates before they are released to everyone, in addition to our internal testing.

So, here are new update candidates, related to the latest Xen security advisories, for XCP-ng 8.0 and 8.1.

Install them with

yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing

and reboot.

Your host is just supposed to keep working normally as before.

Thanks in advance.

HeMaN

@stormi
Did the update and a reboot just minutes ago.
Dell R710 / dual xeon L5640 / 96GB
XCP-ng 8.1 with latest patches / pool with a single host

System is working as expected so far.
If I notice anything during usage I will report back here

[20:41 xenserver ~]# yum update
Loaded plugins: fastestmirror
Determining fastest mirrors
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-base: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-updates: mirrors.xcp-ng.org
dell-system-update_dependent                             | 2.3 kB     00:00
dell-system-update_independent                           | 2.3 kB     00:00
xcp-ng-base/signature                                    |  473 B     00:00
xcp-ng-base/signature                                    | 3.0 kB     00:00 !!!
xcp-ng-updates/signature                                 |  473 B     00:00
xcp-ng-updates/signature                                 | 3.0 kB     00:00 !!!
No packages marked for update
[20:42 xenserver ~]# yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-base: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-testing: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-updates: mirrors.xcp-ng.org
xcp-ng-testing/signature                                 |  473 B     00:00
xcp-ng-testing/signature                                 | 3.0 kB     00:00 !!!
xcp-ng-testing/primary_db                                  |  31 kB   00:00
Resolving Dependencies
--> Running transaction check
---> Package xen-dom0-libs.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-dom0-libs.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
---> Package xen-dom0-tools.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-dom0-tools.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
---> Package xen-hypervisor.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-hypervisor.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
---> Package xen-libs.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-libs.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
---> Package xen-tools.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-tools.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package           Arch      Version                    Repository         Size
================================================================================
Updating:
 xen-dom0-libs     x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing    618 k
 xen-dom0-tools    x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing    1.7 M
 xen-hypervisor    x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing    2.3 M
 xen-libs          x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing     35 k
 xen-tools         x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing     26 k

Transaction Summary
================================================================================
Upgrade  5 Packages

Total download size: 4.7 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/5): xen-dom0-libs-4.13.0-8.6.1.xcpng8.1.x86_64.rpm      | 618 kB   00:00
(2/5): xen-dom0-tools-4.13.0-8.6.1.xcpng8.1.x86_64.rpm     | 1.7 MB   00:00
(3/5): xen-libs-4.13.0-8.6.1.xcpng8.1.x86_64.rpm           |  35 kB   00:00
(4/5): xen-tools-4.13.0-8.6.1.xcpng8.1.x86_64.rpm          |  26 kB   00:00
(5/5): xen-hypervisor-4.13.0-8.6.1.xcpng8.1.x86_64.rpm     | 2.3 MB   00:01
--------------------------------------------------------------------------------
Total                                              3.1 MB/s | 4.7 MB  00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : xen-hypervisor-4.13.0-8.6.1.xcpng8.1.x86_64                 1/10
  Updating   : xen-libs-4.13.0-8.6.1.xcpng8.1.x86_64                       2/10
  Updating   : xen-dom0-libs-4.13.0-8.6.1.xcpng8.1.x86_64                  3/10
  Updating   : xen-tools-4.13.0-8.6.1.xcpng8.1.x86_64                      4/10
  Updating   : xen-dom0-tools-4.13.0-8.6.1.xcpng8.1.x86_64                 5/10
  Cleanup    : xen-dom0-tools-4.13.0-8.5.1.xcpng8.1.x86_64                 6/10
  Cleanup    : xen-tools-4.13.0-8.5.1.xcpng8.1.x86_64                      7/10
  Cleanup    : xen-libs-4.13.0-8.5.1.xcpng8.1.x86_64                       8/10
  Cleanup    : xen-dom0-libs-4.13.0-8.5.1.xcpng8.1.x86_64                  9/10
  Cleanup    : xen-hypervisor-4.13.0-8.5.1.xcpng8.1.x86_64                10/10
  Verifying  : xen-dom0-libs-4.13.0-8.6.1.xcpng8.1.x86_64                  1/10
  Verifying  : xen-libs-4.13.0-8.6.1.xcpng8.1.x86_64                       2/10
  Verifying  : xen-tools-4.13.0-8.6.1.xcpng8.1.x86_64                      3/10
  Verifying  : xen-hypervisor-4.13.0-8.6.1.xcpng8.1.x86_64                 4/10
  Verifying  : xen-dom0-tools-4.13.0-8.6.1.xcpng8.1.x86_64                 5/10
  Verifying  : xen-dom0-libs-4.13.0-8.5.1.xcpng8.1.x86_64                  6/10
  Verifying  : xen-tools-4.13.0-8.5.1.xcpng8.1.x86_64                      7/10
  Verifying  : xen-dom0-tools-4.13.0-8.5.1.xcpng8.1.x86_64                 8/10
  Verifying  : xen-hypervisor-4.13.0-8.5.1.xcpng8.1.x86_64                 9/10
  Verifying  : xen-libs-4.13.0-8.5.1.xcpng8.1.x86_64                      10/10

Updated:
  xen-dom0-libs.x86_64 0:4.13.0-8.6.1.xcpng8.1
  xen-dom0-tools.x86_64 0:4.13.0-8.6.1.xcpng8.1
  xen-hypervisor.x86_64 0:4.13.0-8.6.1.xcpng8.1
  xen-libs.x86_64 0:4.13.0-8.6.1.xcpng8.1
  xen-tools.x86_64 0:4.13.0-8.6.1.xcpng8.1

Complete!
[20:42 xenserver ~]# reboot