XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XCP-ng 8.3 updates announcements and testing

    Scheduled Pinned Locked Moved News
    164 Posts 29 Posters 14.3k Views 41 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G Offline
      Greg_E @Greg_E
      last edited by

      @Greg_E

      No issues so far with my production system (8.2.x) and this batch of important updates, these are on Intel Silver (v2?) processors.

      But my system is so vanilla that I doubt there would be any issues anyway. The only "out of normal" thing I've done today is storage migrate a VM off of one Truenas to another so I can apply some updates to Truenas. Then I'll migrate everything to the updated server so I can update the "main" storage.

      1 Reply Last reply Reply Quote 1
      • P Offline
        ph7
        last edited by

        Can't migrate VHD between pools

        I updated my Intel test host/pool at the New update candidates for you to test! 8 days ago.

        yum clean metadata --enablerepo=xcp-ng-testing
        yum update --enablerepo=xcp-ng-testing
        reboot
        

        I then ran the New security update candidates for you to test! 2 days ago

        yum clean metadata --enablerepo=xcp-ng-candidates
        yum update --enablerepo=xcp-ng-candidates
        reboot
        

        I was no longer able to migrate a VHD from my Intel "test" host/pool to my AMD "production" host/pool
        My thought was I had to wait for the "release" update.

        When the update was published, on my "production" host I ran New update candidates for you to test! and New security update candidates for you to test!
        (why didn't I just run yum update ??)
        Still no migration

        Tried to update my hosts

        [10:23 x1 ~]# yum update
        Inlästa insticksmoduler: fastestmirror
        Loading mirror speeds from cached hostfile
        Excluding mirror: updates.xcp-ng.org
         * xcp-ng-base: mirrors.xcp-ng.org
        Excluding mirror: updates.xcp-ng.org
         * xcp-ng-updates: mirrors.xcp-ng.org
        No packages marked for update
        
        [11:03 x2 ~]# yum update
        Inlästa insticksmoduler: fastestmirror
        Loading mirror speeds from cached hostfile
        Excluding mirror: updates.xcp-ng.org
         * xcp-ng-base: mirrors.xcp-ng.org
        Excluding mirror: updates.xcp-ng.org
         * xcp-ng-updates: mirrors.xcp-ng.org
        No packages marked for update
        

        I then checked the software versions and there is a diff. in
        git_id , date , xapi_build and db_schema

        [10:23 x1 ~]# xe host-list params=software-version
        software-version (MRO)    : product_version: 8.3.0; product_version_text: 8.3; product_version_text_short: 8.3; platform_name: XCP; platform_version: 3.4.0; product_brand: XCP-ng; build_number: 8.3.0; git_id: 0; hostname: localhost; date: 20250507T15:15:51Z; dbv: 0.0.1; xapi: 25.6; xapi_build: 25.6.0; xen: 4.17.5-13; linux: 4.19.0+1; xencenter_min: 2.21; xencenter_max: 2.21; network_backend: openvswitch; db_schema: 5.786
        
        [10:20 x2 ~]# xe host-list params=software-version
        software-version (MRO)    : product_version: 8.3.0; product_version_text: 8.3; product_version_text_short: 8.3; platform_name: XCP; platform_version: 3.4.0; product_brand: XCP-ng; build_number: 8.3.0; git_id: 2; hostname: localhost; date: 20250211T18:05:31Z; dbv: 0.0.1; xapi: 24.19; xapi_build: 24.19.2; xen: 4.17.5-13; linux: 4.19.0+1; xencenter_min: 2.21; xencenter_max: 2.21; network_backend: openvswitch; db_schema: 5.780
        

        What is the next step to fix this?

        P 1 Reply Last reply Reply Quote 0
        • P Offline
          ph7 @ph7
          last edited by

          Now I am not sure if I did run the New update candidates for you to test!
          Is there a way to force the updates?

          gduperreyG 1 Reply Last reply Reply Quote 0
          • gduperreyG Offline
            gduperrey Vates 🪐 XCP-ng Team @ph7
            last edited by

            @ph7 As David mentioned, the security updates were released yesterday. They are no longer in the candidates repository, but in the updates repository.

            Note that the updates in the testing repository have not yet been released. They include a more recent version of the XAPI. This could explain why you can no longer migrate this VHD between your test and production environments.

            Are you trying to perform a live migration or with the VM powered off?

            P 1 Reply Last reply Reply Quote 1
            • P Offline
              ph7 @gduperrey
              last edited by

              @gduperrey said in XCP-ng 8.3 updates announcements and testing:

              Are you trying to perform a live migration or with the VM powered off?

              No live migration, different pools, VDI migration only, powered off.
              Warm migration is working.

              It's OK, I can wait for the release

              A 1 Reply Last reply Reply Quote 0
              • A Offline
                Andrew Top contributor @ph7
                last edited by

                @ph7 @gduperrey With current updates: Cold (VM off) migration works for me. Live migration, when forced due to incompatible CPU fails (badly, host toolstack restart required).

                With the VM off, normal VM/VDI migration worked for me in the following process (VM power on after each migration, and then off again, as a test):

                • XCP 8.2 Intel (Pool 1) -> XCP 8.2 Intel (Pool 2) -> XCP 8.3 Intel (Pool 3) -> XCP 8.3 AMD (Pool 4) -> XCP 8.3 Intel (Pool 3)

                Note: Each host/pool uses local storage. Software versions on hosts match for the same version of XCP. @ph7, looks like you need to yum clean all and yum update again.

                P 1 Reply Last reply Reply Quote 1
                • P Offline
                  ph7 @Andrew
                  last edited by

                  @Andrew

                  looks like you need to yum clean all and yum update again.

                  Tried it but it didn't work
                  And I ran the rm -rf /var/cache/yum

                  [13:09 x2 ~]# yum clean all
                  Inlästa insticksmoduler: fastestmirror
                  Rensar förråd: xcp-ng-base xcp-ng-updates
                  Cleaning up everything
                  Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
                  Cleaning up list of fastest mirrors
                  
                  [13:14 x2 ~]# rm -rf /var/cache/yum
                  
                  [13:14 x2 ~]# yum update
                  Inlästa insticksmoduler: fastestmirror
                  Determining fastest mirrors
                  Excluding mirror: updates.xcp-ng.org
                   * xcp-ng-base: mirrors.xcp-ng.org
                  Excluding mirror: updates.xcp-ng.org
                   * xcp-ng-updates: mirrors.xcp-ng.org
                  xcp-ng-base/signature                     |  473 B  00:00:00     
                  xcp-ng-base/signature                     | 3.0 kB  00:00:00 !!! 
                  xcp-ng-updates/signature                  |  473 B  00:00:00     
                  xcp-ng-updates/signature                  | 3.0 kB  00:00:00 !!! 
                  (1/2): xcp-ng-updates/primary_db          | 238 kB  00:00:00     
                  (2/2): xcp-ng-base/primary_db             | 3.9 MB  00:00:00     
                  No packages marked for update
                  

                  And rebooted

                  P 1 Reply Last reply Reply Quote 0
                  • P Offline
                    ph7 @ph7
                    last edited by

                    Cold migration doesn't work, but Warm migration does.
                    No panic, I can wait for the release

                    1 Reply Last reply Reply Quote 0
                    • stormiS Offline
                      stormi Vates 🪐 XCP-ng Team
                      last edited by

                      Just to be sure, is there an issue for us to investigate here, or expected failure due to version mismatch?

                      P 1 Reply Last reply Reply Quote 0
                      • P Offline
                        ph7 @stormi
                        last edited by

                        @stormi
                        If You are asking me, I can run the New update candidates for you to test! on my "production" home lab server if the testing repo still exist

                        1 Reply Last reply Reply Quote 0
                        • stormiS Offline
                          stormi Vates 🪐 XCP-ng Team
                          last edited by stormi

                          Yes update candidates that were not urgent security fixes are still in the xcp-ng-testing repository (and more is coming soon, today or on
                          monday).

                          P 1 Reply Last reply Reply Quote 0
                          • P Offline
                            ph7 @stormi
                            last edited by

                            @stormi
                            I ran the xcp-ng-testing and now the migration seems to work

                            25871203-4dbe-4e92-ba30-60efb2bf903d-image.png

                            1 Reply Last reply Reply Quote 2
                            • stormiS Offline
                              stormi Vates 🪐 XCP-ng Team
                              last edited by stormi

                              New update update candidates for you to test!

                              Unless major issues are found, this should be the last wave of update candidates before we publish everything as official updates for XCP-ng 8.3.

                              • cifs-utils: update and rebuild based on the sources for the RHEL9 package. This fixes several low priority CVEs (in the context of XCP-ng) and will make future vulnerability patching easier.
                              • curl: update to version 8.9.1, based on RHEL 10 package, and apply an additional fix for CVE-2024-8096 (low impact in XCP-ng context).
                              • intel-e1000e: major driver update, backported from Linux kernel 5.10.179, to fix issues with recent hardware.
                              • kernel: Fix support of dynamic tracepoints when debugging the dom0 Linux kernel with the perf tool
                              • ncurses: Revert -devel package ABI to version 5 to avoid potential library conflicts in packages built against it
                              • openssh: rebuild against updated ncurses package
                              • python3-docutils: new dependency of cifs-utils
                              • samba:
                                • Fix CVE-2016-2124, a flaw on SMB1 auth. An attacker could retrieve the password by using NT1.
                                • Fix CVE-2021-44142, an out-of-bounds heap read write vulnerability that allows remote attackers to execute arbitrary code by using VFS_fruit module.
                              • systemtap: rebuild against updated ncurses package
                              • xapi: Remove pvsproxy.service from the list of units restarted on xcp-rrdd update. The service in question attempts to start a proprietary component from XenServer that isn't present in XCP-ng, which led to displaying a not so pretty error in the logs.
                              • xcp-ng-release: Enable missing xcp-rrdd plugins by default. Yes, failure to do so was what caused the empty stats issue you have been seeing in previous update candidates.
                              • xen: rebuild against updated ncurses package + some fixes.
                              • xo-lite: Update to 0.10.1.

                              Test on XCP-ng 8.3

                              From an up-to-date host:

                              yum clean metadata --enablerepo=xcp-ng-testing
                              yum update --enablerepo=xcp-ng-testing
                              reboot
                              

                              The usual update rules apply: pool coordinator first, etc.

                              Versions

                              • cifs-utils: 7.1-2.1
                              • curl: 8.9.1-5.1.xcpng8.3
                              • intel-e1000e: 5.10.179-1.xcpng8.3
                              • kernel: 4.19.19-8.0.38.2.xcpng8.3
                              • ncurses: 6.4-6.20240309.xcpng8.3
                              • openssh: 7.4p1-23.3.3.xcpng8.3
                              • python3-docutils: 0.14-1.el7
                              • samba: 4.10.16-25.1.xcpng8.3
                              • systemtap: 4.0-5.2.xcpng8.3
                              • xapi: 25.6.0-1.5.xcpng8.3
                              • xcp-ng-release: 8.3.0-32
                              • xen: 4.17.5-10.1.xcpng8.3
                              • xo-lite: 0.10.1-1.xcpng8.3

                              What to test

                              Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

                              Special focus:

                              • We updated the e1000e driver. If you have Intel PCI-Express network chipsets, please test this update and verify that network connectivity and features that you depend on work as expected.
                              • SMB shares and SRs.
                              • yum still appearing to work correctly after the update.
                              • SSH connection to hosts.
                              • Stats. But I'm sure that's the first thing several among you will test already.

                              Test window before official release of the updates

                              Around one week, unless major issues are found.

                              F A P 3 Replies Last reply Reply Quote 0
                              • F Offline
                                flakpyro @stormi
                                last edited by

                                @stormi Updated both of my test hosts.

                                Machine 1:
                                Intel Xeon E-2336
                                SuperMicro board.

                                Machine 2:
                                Minisforum MS-01
                                i9-13900H
                                32 GB Ram
                                Using Intel X710 onboard NIC

                                Everything rebooted and came up fine. The MS-01 i test with uses i40e and intel-igc not the e1000 driver. The other machine with the SuperMicro board uses igb so im afraid i'm not much help in testing that driver.

                                yum commands did seem to work from the small handful i ran.

                                And yes, stats do indeed work again 🙂

                                I never noticed the issue with Server 2025 and hanging on reboot since the updates from last week. Were you able to see anything in the dump files i sent?

                                G stormiS 2 Replies Last reply Reply Quote 2
                                • G Offline
                                  Greg_E @flakpyro
                                  last edited by

                                  @flakpyro

                                  Off topic, how does XCP handle the E and P cores in that 13900 machine? That was one of the considerations that made me skip these Intel processors in favor of AMD.

                                  I'll get mine updated as soon as possible, on going work to clean out my tech room so it can get cut in half for class space. Goodbye future expansion if we need it!

                                  F olivierlambertO 2 Replies Last reply Reply Quote 0
                                  • A Offline
                                    Andrew Top contributor @stormi
                                    last edited by

                                    @stormi Minor issue: e1000e does not show driver version with ethtool:

                                    # ethtool -i eth0
                                    driver: e1000e
                                    version:
                                    firmware-version: 0.1-4
                                    expansion-rom-version:
                                    bus-info: 0000:00:1f.6
                                    supports-statistics: yes
                                    supports-test: yes
                                    supports-eeprom-access: yes
                                    supports-register-dump: yes
                                    supports-priv-flags: yes
                                    
                                    stormiS 1 Reply Last reply Reply Quote 2
                                    • F Offline
                                      flakpyro @Greg_E
                                      last edited by

                                      @Greg_E It seems to handle it fine, i have 2 of these systems running without issue with 8.3!

                                      1 Reply Last reply Reply Quote 1
                                      • stormiS Offline
                                        stormi Vates 🪐 XCP-ng Team @flakpyro
                                        last edited by

                                        @flakpyro No, I'm waiting for more feedback from the devs. All I have from them for now is it looks like either a firmware or a passthrough issue. I don't think we have changed anything to fix it.

                                        1 Reply Last reply Reply Quote 0
                                        • stormiS Offline
                                          stormi Vates 🪐 XCP-ng Team @Andrew
                                          last edited by stormi

                                          @Andrew Thanks for noticing that. CC @ThierryEscande

                                          A 1 Reply Last reply Reply Quote 0
                                          • A Offline
                                            Andrew Top contributor @stormi
                                            last edited by

                                            @stormi @ThierryEscande I issued a PR for intel-e1000e to add the version for ethtool.

                                            stormiS 1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post