XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XCP-ng 8.2 updates announcements and testing

    Scheduled Pinned Locked Moved News
    703 Posts 67 Posters 1.1m Views 86 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stormiS Offline
      stormi Vates 🪐 XCP-ng Team
      last edited by stormi

      New Security Update Candidates (Xen, microcode, ...)

      Components are updated to fix vulnerabilities:

      • Xen is updated to fix XSA-426. It also includes the previous change which had not been released yet: Enable AVX-512 by default for EPYC Zen4 (Genoa)
      • Intel and AMD microcode is updated for various devices:
        • Intel update (which in turn links to the advisories)
        • AMD advisory

      We will also release at the same time:

      • xcp-ng-release-*: fixes benign but annoying fcoe-related error messages at boot

      And an update candidate which has been tested previously:

      • Guest templates: added RHEL 9, AlmaLinux 9, Rocky Linux 9, CentOS Stream 8 & 9, Oracle Linux 9.

      Test on XCP-ng 8.2

      From an up to date host:

      yum clean metadata --enablerepo=xcp-ng-testing
      yum update "guest-templates-*" "xen-*" microcode_ctl linux-firmware "xcp-ng-release-*" --enablerepo=xcp-ng-testing
      reboot
      

      Versions:

      • xen-*: 4.13.4-9.29.2.xcpng8.2
      • microcode_ctl: 2.1-26.xs23.1.xcpng8.2
      • linux-firmware: 20190314-5.1.xcpng8.2
      • guest-templates-json-*: 1.9.6-1.2.xcpng8.2
      • xcp-ng-release-*: 8.2.1-6

      What to test

      Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

      Test window before official release of the updates

      48h

      A M 2 Replies Last reply Reply Quote 1
      • A Online
        Andrew Top contributor @stormi
        last edited by

        @stormi I'm running the update on all 8.2.1 hosts. No problems so far.

        1 Reply Last reply Reply Quote 2
        • olivierlambertO Offline
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          No problem here either on my home lab 🙂

          1 Reply Last reply Reply Quote 1
          • stormiS Offline
            stormi Vates 🪐 XCP-ng Team
            last edited by

            The update was published earlier today: https://xcp-ng.org/blog/2023/02/20/february-2023-security-update/

            R 1 Reply Last reply Reply Quote 2
            • lightingman117L lightingman117 referenced this topic on
            • R Offline
              rjt @stormi
              last edited by

              @stormi @Gaelfr

              I noticed the there are updates to the Windows Templates. Clicking the 👁 or "EYE" in XOA, and the Description for "guest-templates-json-data-windows" seemed a tad smidgeon "buggy". Is that due to git revision description and there were no actual changes to Windows Templates?

              Changelog
              Patch   guest-templates-json-data-windows
              Date    January 6, 2023 at 6:00 AM
              Author Gael Duperrey <gduperrey@vates.fr> - 1.9.6-1.2
              Description   - Add templates for rhel 9, CentOS Stream 8 and 9, Almalinux 9, Rockylinux 9, Oracle linux 9
              
              guest-templates-json	Creates the default guest templates	1.9.6	1.2.xcpng8.2	29.21 KiB	
              guest-templates-json-data-linux	Contains the default Linux guest templates	1.9.6	1.2.xcpng8.2	18.68 KiB	
              guest-templates-json-data-other	Contains the default other guest templates	1.9.6	1.2.xcpng8.2	11.86 KiB	
              guest-templates-json-data-windows	Contains the default Windows guest templates	1.9.6	1.2.xcpng8.2	14.38 KiB
              
              gduperreyG 1 Reply Last reply Reply Quote 0
              • gduperreyG Offline
                gduperrey Vates 🪐 XCP-ng Team @rjt
                last edited by

                @rjt
                These rpms come from the same source rpm and, therefore, from the same SPEC file. So when we build it for changes, the Windows one is built too, even if there is no change on the Windows side.
                On this revision, we only add new templates for RHEL 9, AlmaLinux 9, Rocky Linux 9, CentOS Stream 8 and 9, and Oracle Linux 9.
                There weren't any changes to the Windows templates.

                1 Reply Last reply Reply Quote 1
                • gduperreyG Offline
                  gduperrey Vates 🪐 XCP-ng Team
                  last edited by gduperrey

                  New Security Update Candidates (Xen)

                  Xen is being updated to mitigate some vulnerabilities:

                  • XSA-427: "Guests running in shadow mode and being subject to migration or snapshotting may be able to cause Denial of Service and other problems, including escalation of privilege". This vulnerability concerns old platforms (Nehalem/Bulldozer families and older) which do not have Hardware Assisted Paging facilitie (EPT/NPT), or modern platforms where this extension is disabled by the firmware or the system software. This also concerns PV guests, which are not officially supported anymore in XCP-ng.

                  • XSA-428: "Entities controlling HVM guests can run the host out of resources or stall execution of a physical CPU for effectively unbounded periods of time, resulting in a Denial of Servis (DoS) affecting the entire host. Crashes, information leaks, or elevation of privilege cannot be ruled out".
                    On the platforms managed by XCP-ng software, with regard of this vulnerability, we would rather talk of "reduction in defence in depth", as the only entity controlling HVM guests is a trusted software (QEMU) running in a trusted domain (dom0).

                  • XSA-429: The patch completes the original Spectre/Meltdown mitigation work(XSA-254). A malicious PV guest might be able to infer the contents of arbitrary host memory, including memory assigned to other guests. Only AMD and Hygon CPUs which offer SMEP/SMAP facilities are affected. Although PV guests are not officially supported in XCP-ng, we also included a fix for this vulnerability.

                  Components are also updated to add bugfixes and enhancements:

                  • Xen
                    • Update to Xen 4.13.5
                    • Initial Sapphire Rapids support
                    • Fix memory corruption issues in the Ocaml bindings.
                    • On xenstored live update, validate the config file before launching into the new xenstored

                  Test on XCP-ng 8.2

                  From an up to date host:

                  yum clean metadata --enablerepo=xcp-ng-testing
                  yum update "xen-*" --enablerepo=xcp-ng-testing
                  reboot
                  

                  Versions:

                  • xen-*: 4.13.5-9.30.3.xcpng8.2

                  What to test

                  Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

                  Test window before official release of the updates

                  ~2 days.

                  A 1 Reply Last reply Reply Quote 1
                  • A Online
                    Andrew Top contributor @gduperrey
                    last edited by

                    @gduperrey Installed on 8.2 systems and running ok on home lab and other secondary machines. No issues before or after update. No Sapphire Rapids CPUs. Ran same update on 8.3 by mistake on a test machine but it's running ok too.

                    1 Reply Last reply Reply Quote 2
                    • M Offline
                      maxcuttins @stormi
                      last edited by

                      @stormi

                      what is the kernel version of the latest XCP-ng?

                      stormiS 1 Reply Last reply Reply Quote 0
                      • stormiS Offline
                        stormi Vates 🪐 XCP-ng Team
                        last edited by

                        Looks like some of the testers who used to test the update candidates moved their test hosts to the 8.3 alpha release. Thanks @Andrew for staying true to the job 🙂

                        1 Reply Last reply Reply Quote 0
                        • stormiS Offline
                          stormi Vates 🪐 XCP-ng Team @maxcuttins
                          last edited by

                          @maxcuttins Is the question related to the testing of update candidates?

                          M 1 Reply Last reply Reply Quote 0
                          • J Offline
                            JeffBerntsen Top contributor
                            last edited by

                            Latest updates seem to be working fine in my test lab as well.

                            1 Reply Last reply Reply Quote 2
                            • gduperreyG Offline
                              gduperrey Vates 🪐 XCP-ng Team
                              last edited by

                              Update published. Thanks for the tests!

                              https://xcp-ng.org/blog/2023/03/23/march-2023-security-update/

                              A 1 Reply Last reply Reply Quote 1
                              • M Offline
                                maxcuttins @stormi
                                last edited by

                                @stormi not really I would say.
                                I'm stick to version 8.0 and I'm planning to upgrade.
                                So I'm trying to understand which kernel I'll find in the next release.

                                PS: Upgrade is a pain because I need to remember that CEPH NBD share storage are not preserved during upgrade and so, those config file will be erased, I'll need to restore in order to have back my VDIs.

                                stormiS 1 Reply Last reply Reply Quote 0
                                • stormiS Offline
                                  stormi Vates 🪐 XCP-ng Team @maxcuttins
                                  last edited by

                                  @maxcuttins Ok. Then let's discuss this in another thread and leave the current one for testing update candidates.

                                  1 Reply Last reply Reply Quote 0
                                  • A Online
                                    Andrew Top contributor @gduperrey
                                    last edited by

                                    @gduperrey XO (current source) rolling pool update did its job.

                                    1 Reply Last reply Reply Quote 3
                                    • stormiS Offline
                                      stormi Vates 🪐 XCP-ng Team
                                      last edited by

                                      Hello here! I hope you are ready, because we'll have a train of update candidates for you to test shortly 🙂

                                      1 Reply Last reply Reply Quote 5
                                      • stormiS Offline
                                        stormi Vates 🪐 XCP-ng Team
                                        last edited by stormi

                                        New update candidates for you to test!

                                        As you may know, we group non-urgent updates together for a collective release, in order not to cause unnecessary maintenance for our users.

                                        The moment to release such a batch has come, so here they are, ready for user tests before the final release.

                                        • xcp-ng-release*:
                                          • Updated web page on hosts to remove dependencies to Fontawesome Pro and Jquery.
                                          • The XOA quick deploy script now uses HTTPS by default.
                                          • Updated repository definitions in /etc/yum.repos.d/xcpng.repo, to add more testing repositories (disabled by default. More about this below). Warning: If you have any local changes to this file, it won't be overwritten. In this case, look for /etc/yum.repos.d/xcpng.repo.rpmnew after applying the update, and move it over xcpng.repo.
                                        • xen-*: sync with Citrix Hypervisor hotfix XS82ECU1030:
                                          • Hardware support fixes, among which "Cope booting for x2APIC mode on AMD systems without XT mode."
                                          • Improve loading of AMD microcode on all logical processors.
                                          • (The hotfix from Citrix Hypervisor also includes fixes for the latest Xen Security Advisories, which we already published in a previous update)
                                        • AMD microcode (linux-firmware) and Intel microcode (microcode_ctl). AMD and Intel did not detail what they fix, but everyone is supposed to update. This is the frustrating situation with binary blobs in firmware.
                                        • XAPI and related components:
                                          • Instead of a 403 error on HTTP requests to the host's web page, redirect to HTTPS instead.
                                          • Fix spurious "not enough memory" error message in /var/log/xcp-rrdd-plugins.log.
                                          • Sync with Citrix Hypervisor hotfix XS82ECU1027: various fixes.
                                        • qemu: sync with Citrix Hypervisor hotfix XS82ECU1031. Fixes for specific issues.
                                        • sm (Storage Manager): sync with Citrix hypervisor hotfix XS82ECU1022. Various fixes.

                                        Test on XCP-ng 8.2

                                        yum clean metadata --enablerepo=xcp-ng-testing
                                        yum update --enablerepo=xcp-ng-testing forkexecd gpumon linux-firmware message-switch microcode_ctl qemu rrdd-plugins sm sm-rawhba varstored-guard xapi-core xapi-tests xapi-xe xcp-networkd xcp-ng-release xcp-ng-release-config xcp-ng-release-presets xcp-rrdd xen-dom0-libs xen-dom0-tools xen-hypervisor xen-libs xen-tools xenopsd xenopsd-cli xenopsd-xc
                                        reboot
                                        

                                        The usual update rules apply: pool coordinator first, etc.

                                        What to test

                                        Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

                                        About the new testing repositories

                                        Until recently, we would just have one testing repository: xcp-ng-testing. We decided to split it.

                                        It had a lot of different uses:

                                        • Making updates available to testers for them to provide feedback, before pushing them to everyone. This use will remain and will now be the only role of this repository.
                                        • Storing updates to components we don't intend to push as official updates. For example newer zstd or GlusterFS releases. These now live in a new repository: xcp-ng-lab.
                                        • Providing temporary builds just to test a patch, before embedding it in a real update if tests are successful. There are several places where we can make them available to you when needed, depending on the situation: per-person developer repositories, scratch builds in koji, or . We'll tell you where to pull from each time we need you to test.

                                        We also added two new repositories for our internal needs. You usually won't need to pull from them, even for tests: xcp-ng-incoming and xcp-ng-ci. xcp-ng-incoming is where we build updates first. When a consistent set of changes is ready, it moves to xcp-ng-ci and undergoes automated testing. Once the tests pass,
                                        updates move to xcp-ng-testing for you to test.

                                        Shortly before publishing to everyone, updates will be moved the new xcp-ng-candidates. Why are there both xcp-ng-testing and xcp-ng-candidates? Because not all updates move on at the same pace. Some can wait for weeks before we publish them in what we call internally "an update train", because we group non-urgent updates together. Some need to be published as soon as possible, notably security fixes. So while there may already be updates in xcp-ng-testing, sometimes we need to build, test and publish updates directly without any interferences from what's currently in xcp-ng-testing. What it means for you as testers is that sometimes we'll ask you to pull update candidates from xcp-ng-testing, sometimes from xcp-ng-candidates. In any case we'll always specify it in our testing instructions.

                                        Test window before official release of the updates

                                        ~1 week.

                                        A J 2 Replies Last reply Reply Quote 2
                                        • stormiS Offline
                                          stormi Vates 🪐 XCP-ng Team
                                          last edited by

                                          We had some feedback on 8.3, but I'm also counting on you for XCP-ng 8.2 😉

                                          1 Reply Last reply Reply Quote 0
                                          • A Online
                                            Andrew Top contributor @stormi
                                            last edited by Andrew

                                            @stormi I updated active 8.2.1 servers and it's running normally (24 hours), HP G8 and current 11th Gen i7. I updated other machines (older AMD and Intel) and they are ok too, but just used for testing. Normal update/reboot worked fine.

                                            Active servers run: Windows, Linux (many versions), FreeBSD, hot migrations, CR, Delta S3 backup, NFS SR/ISO, VxLAN, etc...

                                            1 Reply Last reply Reply Quote 2
                                            • First post
                                              Last post