XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XCP-ng 8.2 updates announcements and testing

    Scheduled Pinned Locked Moved
    News
    67
    680
    273.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stormiS
      stormi Vates 🪐 XCP-ng Team
      last edited by stormi

      First update for XCP-ng 8.1:

      • xcp-ng-deps for pulling the gpumon package. That issue has kept us busy this week when people started reporting installed or upgraded systems that would be unreachable from XO and XCP-ng Center after a reboot. Turns out that XAPI needs gpumon, a package that can only be built using a proprietary (from what I remember. One would need to check the license) nVIDIA developement toolkit. I had removed it from XCP-ng 8.1 because I was convinced that it was only necessary with the vgpu feature with nVIDIA, and we don't have the vgpu package in XCP-ng because it is closed-source. XAPI's start process stalls without an error message if there's an nVIDIA GPU. Installing gpumon fixes it.
        • after the update: if you were affected by the issue (you would know), you may need to do an emergency network restart and possibly reconnect the SRs. There may be other consequences for the hosts, such as missing removable media from VMs and possibly others. If you have a way to come back to a backup and reinstall with the fixed ISO (released 2020-04-06), this may be the safest path.
        • see also https://github.com/xcp-ng/xcp/wiki/XCP-ng-8.1-Known-Issues#host-unreachable-after-upgrade-or-fresh-installation-on-hosts-having-an-nvidia-gpu
      • xcp-ng-release* for reducing chrony-wait's timeout from 600s to 120s. So if your host can't connect the ntp server that was configured at installation, you'll only have to wait for 2 minutes, not 10. But your hosts really should be able to connect a ntp server. Accurate time is required to avoid timing and synchronisation issues.

      No reboot required if your host is already behaving correctly. If you have a discrete nVIDIA GPU and the host had no issue, then 1. I'm surprised, 2. I advise to reboot.

      New installation ISOs including those two updates will be released shortly. 2020-04-06 update: they have been released, named xcp-ng-8.1.0-2.iso and xcp-ng-8.1.0-2-netinstall.iso.

      As ususal, see https://github.com/xcp-ng/xcp/wiki/Updates-Howto

      M 1 Reply Last reply Reply Quote 4
      • M
        marekm @stormi
        last edited by

        @stormi Is this related to https://bugs.xenserver.org/browse/XSO-936 (build dependency on gdk-devel) - or a different issue? (I suspect gdk-devel is some part of GNOME, not nVidia proprietary - but could be wrong.)

        stormiS 1 Reply Last reply Reply Quote 0
        • stormiS
          stormi Vates 🪐 XCP-ng Team @marekm
          last edited by

          @marekm said in Updates announcements and testing:

          @stormi Is this related to https://bugs.xenserver.org/browse/XSO-936 (build dependency on gdk-devel) - or a different issue? (I suspect gdk-devel is some part of GNOME, not nVidia proprietary - but could be wrong.)

          Yes it is this issue, and no, it's not GNOME's gdk-devel. Here GDK stands for GPU Deployment Kit. It's unfortunate that someone at some point thought that it would be a good idea to name the dependency gdk-devel, which was already taken by the GNOME project.

          M 1 Reply Last reply Reply Quote 0
          • stormiS
            stormi Vates 🪐 XCP-ng Team
            last edited by

            By the way, if someone wants to have a look at how to build gpumon from its source RPM and prove me wrong about the impossibility to do it only with FOSS tools, I'll gladly revise my judgment 🙂

            1 Reply Last reply Reply Quote 0
            • M
              marekm @stormi
              last edited by

              @stormi OK, just asked since the issue is about a year old and still unanswered by Citrix, wondering why it didn't show up in earlier xcp-ng releases. Perhaps they have only recently made some change to XAPI so it requires gpumon, and this change could be reverted (or better yet, fixed to handle the missing package gracefully, not hang the whole machine at boot time).

              stormiS 1 Reply Last reply Reply Quote 0
              • stormiS
                stormi Vates 🪐 XCP-ng Team @marekm
                last edited by

                @marekm said in Updates announcements and testing:

                @stormi OK, just asked since the issue is about a year old and still unanswered by Citrix, wondering why it didn't show up in earlier xcp-ng releases. Perhaps they have only recently made some change to XAPI so it requires gpumon, and this change could be reverted (or better yet, fixed to handle the missing package gracefully, not hang the whole machine at boot time).

                It's simpler: it's only with XCP-ng 8.1 that I have removed gpumon. That was a mistake so I had to put it back.

                M 1 Reply Last reply Reply Quote 0
                • M
                  marekm @stormi
                  last edited by

                  @stormi Thanks. Is this https://github.com/xenserver/gpumon - if yes, it doesn't look like something big, perhaps could be patched to remove dependency on proprietary nvidia stuff.

                  1 Reply Last reply Reply Quote 0
                  • stormiS
                    stormi Vates 🪐 XCP-ng Team
                    last edited by

                    Yes, it's this. I've also create an issue for the XAPI project, that got good reception from the devs. All agree that XAPI should be able to start even if gpumon is not there.

                    1 Reply Last reply Reply Quote 0
                    • stormiS
                      stormi Vates 🪐 XCP-ng Team
                      last edited by stormi

                      Note that patching gpumon so that it doesn't depend on nVIDIA stuff probably just means making it a stub service that always answers the same regardless of the actual state of the GPUs.

                      1 Reply Last reply Reply Quote 0
                      • stormiS
                        stormi Vates 🪐 XCP-ng Team
                        last edited by

                        Following yesterday's Xen security updates released by Citrix, here are test packages with security patches. As usual with security matters, they need to be tested quickly so that we can release them to everyone fast.

                        On XCP-ng 8.1

                        # on an up to date host
yum clean all --enablerepo=xcp-ng-testing
yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing

                        reboot (master first)

                        On XCP-ng 8.0

                        This batch of updates contains not only security fixes but also bug fixes that I had queued for the next patch train.

                        # on an up to date host
yum clean all --enablerepo=xcp-ng-testing
yum update bugtool-conn-tests qlogic-netxtreme2 qlogic-netxtreme2-4.19.0+1-modules sm sm-rawhba xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools xenserver-status-report --enablerepo=xcp-ng-testing

                        reboot (master first)

                        On XCP-ng 7.6

                        XCP-ng 7.6 is not supported anymore, so there are no updates available for it.

                        1 Reply Last reply Reply Quote 1
                        • stormiS
                          stormi Vates 🪐 XCP-ng Team
                          last edited by

                          Hi users!

                          Today's the last day to install the update candidate and give feedback. If you have any test host, please install and reboot. That's it and it's useful to us.

                          1 Reply Last reply Reply Quote 1
                          • stormiS
                            stormi Vates 🪐 XCP-ng Team
                            last edited by

                            Update released: April 2020 XCP-ng Security Updates

                            1 Reply Last reply Reply Quote 0
                            • stormiS
                              stormi Vates 🪐 XCP-ng Team
                              last edited by

                              Intel CPUs "CROSSTalk" vulnerability.

                              Following the disclosure of the CROSSTalk CPU vulnerabilities and the release of updated microcode by Intel, here are update candidates for XCP-ng 8.0 and 8.1. Prompt feedback by all available testers is wanted.

                              Details and discussion in the dedicated thread.

                              1 Reply Last reply Reply Quote 1
                              • stormiS
                                stormi Vates 🪐 XCP-ng Team
                                last edited by

                                Update published: https://xcp-ng.org/blog/2020/06/12/intel-microcode-security-update-crosstalk/

                                1 Reply Last reply Reply Quote 0
                                • stormiS
                                  stormi Vates 🪐 XCP-ng Team
                                  last edited by

                                  I haven't had many tests from the community for the previous updates.

                                  I'm still trying to convince myself that I can count on the community to test updates before they are released to everyone, in addition to our internal testing.

                                  So, here are new update candidates, related to the latest Xen security advisories, for XCP-ng 8.0 and 8.1.

                                  Install them with

                                  yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing

                                  and reboot.

                                  Your host is just supposed to keep working normally as before.

                                  Thanks in advance.

                                  H 1 Reply Last reply Reply Quote 2
                                  • H
                                    HeMaN @stormi
                                    last edited by

                                    @stormi
                                    Did the update and a reboot just minutes ago.
                                    Dell R710 / dual xeon L5640 / 96GB
                                    XCP-ng 8.1 with latest patches / pool with a single host

                                    System is working as expected so far.
                                    If I notice anything during usage I will report back here

                                    [20:41 xenserver ~]# yum update
Loaded plugins: fastestmirror
Determining fastest mirrors
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-base: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-updates: mirrors.xcp-ng.org
dell-system-update_dependent                             | 2.3 kB     00:00
dell-system-update_independent                           | 2.3 kB     00:00
xcp-ng-base/signature                                    |  473 B     00:00
xcp-ng-base/signature                                    | 3.0 kB     00:00 !!!
xcp-ng-updates/signature                                 |  473 B     00:00
xcp-ng-updates/signature                                 | 3.0 kB     00:00 !!!
No packages marked for update
[20:42 xenserver ~]# yum update xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools --enablerepo=xcp-ng-testing
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-base: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-testing: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-updates: mirrors.xcp-ng.org
xcp-ng-testing/signature                                 |  473 B     00:00
xcp-ng-testing/signature                                 | 3.0 kB     00:00 !!!
xcp-ng-testing/primary_db                                  |  31 kB   00:00
Resolving Dependencies
--> Running transaction check
---> Package xen-dom0-libs.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-dom0-libs.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
---> Package xen-dom0-tools.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-dom0-tools.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
---> Package xen-hypervisor.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-hypervisor.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
---> Package xen-libs.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-libs.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
---> Package xen-tools.x86_64 0:4.13.0-8.5.1.xcpng8.1 will be updated
---> Package xen-tools.x86_64 0:4.13.0-8.6.1.xcpng8.1 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package           Arch      Version                    Repository         Size
================================================================================
Updating:
 xen-dom0-libs     x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing    618 k
 xen-dom0-tools    x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing    1.7 M
 xen-hypervisor    x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing    2.3 M
 xen-libs          x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing     35 k
 xen-tools         x86_64    4.13.0-8.6.1.xcpng8.1      xcp-ng-testing     26 k

Transaction Summary
================================================================================
Upgrade  5 Packages

Total download size: 4.7 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/5): xen-dom0-libs-4.13.0-8.6.1.xcpng8.1.x86_64.rpm      | 618 kB   00:00
(2/5): xen-dom0-tools-4.13.0-8.6.1.xcpng8.1.x86_64.rpm     | 1.7 MB   00:00
(3/5): xen-libs-4.13.0-8.6.1.xcpng8.1.x86_64.rpm           |  35 kB   00:00
(4/5): xen-tools-4.13.0-8.6.1.xcpng8.1.x86_64.rpm          |  26 kB   00:00
(5/5): xen-hypervisor-4.13.0-8.6.1.xcpng8.1.x86_64.rpm     | 2.3 MB   00:01
--------------------------------------------------------------------------------
Total                                              3.1 MB/s | 4.7 MB  00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : xen-hypervisor-4.13.0-8.6.1.xcpng8.1.x86_64                 1/10
  Updating   : xen-libs-4.13.0-8.6.1.xcpng8.1.x86_64                       2/10
  Updating   : xen-dom0-libs-4.13.0-8.6.1.xcpng8.1.x86_64                  3/10
  Updating   : xen-tools-4.13.0-8.6.1.xcpng8.1.x86_64                      4/10
  Updating   : xen-dom0-tools-4.13.0-8.6.1.xcpng8.1.x86_64                 5/10
  Cleanup    : xen-dom0-tools-4.13.0-8.5.1.xcpng8.1.x86_64                 6/10
  Cleanup    : xen-tools-4.13.0-8.5.1.xcpng8.1.x86_64                      7/10
  Cleanup    : xen-libs-4.13.0-8.5.1.xcpng8.1.x86_64                       8/10
  Cleanup    : xen-dom0-libs-4.13.0-8.5.1.xcpng8.1.x86_64                  9/10
  Cleanup    : xen-hypervisor-4.13.0-8.5.1.xcpng8.1.x86_64                10/10
  Verifying  : xen-dom0-libs-4.13.0-8.6.1.xcpng8.1.x86_64                  1/10
  Verifying  : xen-libs-4.13.0-8.6.1.xcpng8.1.x86_64                       2/10
  Verifying  : xen-tools-4.13.0-8.6.1.xcpng8.1.x86_64                      3/10
  Verifying  : xen-hypervisor-4.13.0-8.6.1.xcpng8.1.x86_64                 4/10
  Verifying  : xen-dom0-tools-4.13.0-8.6.1.xcpng8.1.x86_64                 5/10
  Verifying  : xen-dom0-libs-4.13.0-8.5.1.xcpng8.1.x86_64                  6/10
  Verifying  : xen-tools-4.13.0-8.5.1.xcpng8.1.x86_64                      7/10
  Verifying  : xen-dom0-tools-4.13.0-8.5.1.xcpng8.1.x86_64                 8/10
  Verifying  : xen-hypervisor-4.13.0-8.5.1.xcpng8.1.x86_64                 9/10
  Verifying  : xen-libs-4.13.0-8.5.1.xcpng8.1.x86_64                      10/10

Updated:
  xen-dom0-libs.x86_64 0:4.13.0-8.6.1.xcpng8.1
  xen-dom0-tools.x86_64 0:4.13.0-8.6.1.xcpng8.1
  xen-hypervisor.x86_64 0:4.13.0-8.6.1.xcpng8.1
  xen-libs.x86_64 0:4.13.0-8.6.1.xcpng8.1
  xen-tools.x86_64 0:4.13.0-8.6.1.xcpng8.1

Complete!
[20:42 xenserver ~]# reboot
                                    1 Reply Last reply Reply Quote 2
                                    • gskgerG
                                      gskger Top contributor
                                      last edited by

                                      Same here.
                                      Updated a Dell R720 / dual Intel Xeon CPU E5-2640 v2 / 64GB
                                      XCP-ng 8,1 fully patched
                                      Standalone host

                                      Still have to do some testing on / with VMs, but this is something for the weekend.

                                      1 Reply Last reply Reply Quote 1
                                      • stormiS
                                        stormi Vates 🪐 XCP-ng Team
                                        last edited by

                                        Update published. Thanks for testing!

                                        https://xcp-ng.org/blog/2020/07/10/july-2020-xcp-ng-security-updates/

                                        1 Reply Last reply Reply Quote 1
                                        • stormiS
                                          stormi Vates 🪐 XCP-ng Team
                                          last edited by

                                          Hi everyone.

                                          An update candidate is available for netdata packages in XCP-ng 8.0 and 8.1. Feedback about it is welcome.

                                          They fix two issues:

                                          • a buffer overflow in JSON parsing, that could pose a security threat
                                          • netdata logs being flooded due to a bug in the computation of the last vcpu number

                                          To install the update candidate on XCP-ng 8.0 or 8.1:

                                          yum update "netdata*" --enablerepo=xcp-ng-testing

                                          This will update the netdata and netdata-ui RPMs if they are installed on the host.

                                          You can also install them if not already present with:

                                          yum install netdata-ui --enablerepo=xcp-ng-testing

                                          Note that installing netdata-ui automatically opens port 19999.

                                          1 Reply Last reply Reply Quote 0
                                          • antoniolfdacruzA
                                            antoniolfdacruz
                                            last edited by 

                                            [10:37 xcpngp02h01 ~]# yum update "netdata*" --enablerepo=xcp-ng-testing
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-base: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-testing: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
 * xcp-ng-updates: mirrors.xcp-ng.org
xcp-ng-testing/signature                                 |  473 B     00:00     
xcp-ng-testing/signature                                 | 3.0 kB     00:00 !!! 
xcp-ng-testing/primary_db                                  |  22 kB   00:01     
Resolving Dependencies
--> Running transaction check
---> Package netdata.x86_64 0:1.19.0-3.xcpng8.1 will be updated
---> Package netdata.x86_64 0:1.19.0-4.xcpng8.1 will be an update
---> Package netdata-debuginfo.x86_64 0:1.19.0-3.xcpng8.1 will be updated
---> Package netdata-debuginfo.x86_64 0:1.19.0-4.xcpng8.1 will be an update
---> Package netdata-ui.x86_64 0:1.19.0-3.xcpng8.1 will be updated
---> Package netdata-ui.x86_64 0:1.19.0-4.xcpng8.1 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package              Arch      Version                 Repository         Size
================================================================================
Updating:
 netdata              x86_64    1.19.0-4.xcpng8.1       xcp-ng-testing     11 M
 netdata-debuginfo    x86_64    1.19.0-4.xcpng8.1       xcp-ng-testing    1.6 M
 netdata-ui           x86_64    1.19.0-4.xcpng8.1       xcp-ng-testing    6.4 k

Transaction Summary
================================================================================
Upgrade  3 Packages

Total download size: 13 M
Is this ok [y/d/N]: y
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
(1/3): netdata-debuginfo-1.19.0-4.xcpng8.1.x86_64.rpm      | 1.6 MB   00:02     
(2/3): netdata-ui-1.19.0-4.xcpng8.1.x86_64.rpm             | 6.4 kB   00:00     
(3/3): netdata-1.19.0-4.xcpng8.1.x86_64.rpm                |  11 MB   00:12     
--------------------------------------------------------------------------------
Total                                              1.1 MB/s |  13 MB  00:12     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : netdata-1.19.0-4.xcpng8.1.x86_64                             1/6 
  Updating   : netdata-ui-1.19.0-4.xcpng8.1.x86_64                          2/6 
  Updating   : netdata-debuginfo-1.19.0-4.xcpng8.1.x86_64                   3/6 
  Cleanup    : netdata-ui-1.19.0-3.xcpng8.1.x86_64                          4/6 
  Cleanup    : netdata-debuginfo-1.19.0-3.xcpng8.1.x86_64                   5/6 
  Cleanup    : netdata-1.19.0-3.xcpng8.1.x86_64                             6/6 
  Verifying  : netdata-debuginfo-1.19.0-4.xcpng8.1.x86_64                   1/6 
  Verifying  : netdata-1.19.0-4.xcpng8.1.x86_64                             2/6 
  Verifying  : netdata-ui-1.19.0-4.xcpng8.1.x86_64                          3/6 
  Verifying  : netdata-ui-1.19.0-3.xcpng8.1.x86_64                          4/6
  Verifying  : netdata-1.19.0-3.xcpng8.1.x86_64                             5/6 
  Verifying  : netdata-debuginfo-1.19.0-3.xcpng8.1.x86_64                   6/6 

Updated:
  netdata.x86_64 0:1.19.0-4.xcpng8.1                                            
  netdata-debuginfo.x86_64 0:1.19.0-4.xcpng8.1                                  
  netdata-ui.x86_64 0:1.19.0-4.xcpng8.1                                         

Complete!
[12:47 xcpngp02h01 ~]# 
[12:50 xcpngp02h01 ~]# systemctl status netdata.service 
? netdata.service - Real time performance monitoring
   Loaded: loaded (/usr/lib/systemd/system/netdata.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2020-07-16 12:50:46 -03; 10s ago
  Process: 9810 ExecStartPre=/usr/libexec/netdata/xcpng-iptables-restore.sh (code=exited, status=0/SUCCESS)
  Process: 9807 ExecStartPre=/bin/chown -R netdata:netdata /var/run/netdata (code=exited, status=0/SUCCESS)
  Process: 9804 ExecStartPre=/bin/mkdir -p /var/run/netdata (code=exited, status=0/SUCCESS)
  Process: 9800 ExecStartPre=/bin/chown -R netdata:netdata /var/cache/netdata (code=exited, status=0/SUCCESS)
  Process: 9797 ExecStartPre=/bin/mkdir -p /var/cache/netdata (code=exited, status=0/SUCCESS)
 Main PID: 9816 (netdata)
   CGroup: /system.slice/netdata.service
           ??9816 /usr/sbin/netdata -P /var/run/netdata/netdata.pid -D -W set...
           ??9849 /usr/bin/python /usr/libexec/netdata/plugins.d/python.d.plu...
           ??9852 /usr/libexec/netdata/plugins.d/go.d.plugin 1
           ??9854 /usr/libexec/netdata/plugins.d/freeipmi.plugin 1
           ??9857 /usr/libexec/netdata/plugins.d/xenstat.plugin 1
           ??9867 /usr/libexec/netdata/plugins.d/apps.plugin 1
[12:50 xcpngp02h01 ~]#
                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post