XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    XCP-ng 8.3 updates announcements and testing

    Scheduled Pinned Locked Moved News
    123 Posts 25 Posters 8.0k Views 40 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G Offline
      Greg_E @stormi
      last edited by

      @stormi

      My master host hung on reboot, after about 10 minutes I forced the power off, and then back on. Host is HP T740 Thinclient with AMD v1756b processor, 64GB of DDR4 SODIMM, Intel dual x520 PCIe card, and an Intel i226-v in the a+e slot, plus the onboard Realtek NIC, BIOS at 1.20 which I think is still current for this model. All three hosts are identical with possible exception of x520 card revisions, host 3 might have an older revision (donations accepted for x710 based cards 😀 )

      Otherwise everything went as planned, all three are updated and all three needed to have the toolstack restarted to see the stats. I only have 2 small Linux VMs on this system right now, and both of them started fine.

      There were no stats in XO-Lite either, I was doing the second and third hosts from XO-Lite to see if XO was getting mad. The second and third rebooted without issue and something I'll look into with the next update, the delay on master could have been the VMs trying to auto start where I remembered to turn that off for the other two hosts.

      1 Reply Last reply Reply Quote 1
      • B Offline
        bufanda
        last edited by

        Installed on my Test pool with 2 HP EliteDesk 800 G3 mini. Except the no stats in XenOrchestra. Did some VM migragration between hosts, reboot testes (only Linux VMs though) and so far no issues.

        1 Reply Last reply Reply Quote 2
        • A Offline
          andriy.sultanov Vates 🪐 XAPI & Network Team @abudef
          last edited by andriy.sultanov

          @abudef @Greg_E @bufanda

          Could you please attach /var/log/{xensource.log,daemon.log,xcp-rrdd-plugins.log} from the time after the reboot and before the toolstack restart?

          Also: does this reproduce if you reboot again - would you still have no metrics until toolstack restart? Because if not, this is entering paranormal territory 🙂

          UPD: I've just updated xapi from 24.39 to 25.6 myself - stats are working after the reboot and I can see them in XenOrchestra, no additional toolstack restart required.

          abudefA B 2 Replies Last reply Reply Quote 0
          • abudefA Offline
            abudef @andriy.sultanov
            last edited by

            @andriy.sultanov said in XCP-ng 8.3 updates announcements and testing:

            does this reproduce if you reboot again

            Yes, this does. Logs have been provided via pm.

            1 Reply Last reply Reply Quote 0
            • B Offline
              bufanda @andriy.sultanov
              last edited by

              @andriy.sultanov
              Same issue after another reboot. The stats on host just flatline. No CPU usage or anything.
              See screenshot
              f58c86b0-ac72-4a6c-a554-ecee918e4db7-image.png

              I attached log files
              daemon.log.txt xcp-rrdd-plugins.log.txt xensource.log.txt

              A 1 Reply Last reply Reply Quote 0
              • A Offline
                andriy.sultanov Vates 🪐 XAPI & Network Team @bufanda
                last edited by andriy.sultanov

                @bufanda Thanks!
                Hmm, can't see anything suspicious either in your logs or @abudef's

                Since you said it reproduces if you reboot the host again, I'd really appreciate if you could send the result of rrd2csv running for 30 seconds or so on a system that doesn't have the stats working - so before any toolstack restarts.

                B G 2 Replies Last reply Reply Quote 0
                • B Offline
                  bufanda @andriy.sultanov
                  last edited by

                  @andriy.sultanov
                  I ran it for a minute.

                  rrd2csv.txt

                  1 Reply Last reply Reply Quote 0
                  • G Offline
                    Greg_E @andriy.sultanov
                    last edited by

                    @andriy.sultanov

                    If still needed, I can probably get you the rrd2csv next week, I need to unrack my system and move it to a new mobile rack. A large expense that I'm not happy to have to do, but now needed going forward (it's a work thing, where I prototype workflows for my production system) 😠 😡 😣

                    A 1 Reply Last reply Reply Quote 0
                    • A Offline
                      andriy.sultanov Vates 🪐 XAPI & Network Team @Greg_E
                      last edited by

                      @Greg_E Thanks, but that will not be necessary - I think I've figured out where the problem lies now. Good luck with the move 🙂

                      G 1 Reply Last reply Reply Quote 1
                      • G Offline
                        Greg_E @andriy.sultanov
                        last edited by

                        @andriy.sultanov

                        The move is more about spending yet more personal money for something that is primarily used for learning what I need to know for work (a rack, UPS, and some odds and ends at $800). Or half of it is, half of it is a VMware system and that's to let me learn what I need to leave this job and find one that pays more. And then move that work to something easier to use that costs less. Still amazed at how many places just took the price increase and are still not making plans to move to something else, even if they are cutting core counts in half to save half of that new money. But I'm also seeing that the general trend in IT around where I live is to get into a Silo, and never ever take on another task to fill a need. This way you never want to move laterally to other products, just keep doing the same things the same ways until management gets tired of hearing "we can't do that" and fires everyone to replace them with an MSP (or other contractor).

                        So for the off topic rant, feeling salty again today.

                        1 Reply Last reply Reply Quote 0
                        • stormiS Offline
                          stormi Vates 🪐 XCP-ng Team
                          last edited by

                          New security update candidates for you to test!

                          We're still working on the updates that you started testing (and a few more), but right now there's an emergency: a security update.

                          Yet more vulnerabilities in Intel hardware, addressed in two complementary ways: patching Xen and updating Intel microcode.

                          Test on XCP-ng 8.3

                          From an up-to-date host, or a host on which previous update candidates had been installed. Both fit.

                          yum clean metadata --enablerepo=xcp-ng-candidates
                          yum update --enablerepo=xcp-ng-candidates
                          reboot
                          

                          The usual update rules apply: pool coordinator first, etc.

                          Versions

                          • intel-microcode: 20250501-1.xcpng8.3
                          • xen: 4.17.5-13.1.xcpng8.3

                          What to test

                          Normal use and anything else you want to test. The closer to your actual use of XCP-ng, the better.

                          Test window before official release of the updates

                          ~24h. That's an urgent one.

                          P G A 3 Replies Last reply Reply Quote 3
                          • P Offline
                            ph7 @stormi
                            last edited by

                            @stormi
                            Stats still not good
                            Only rebooted
                            a9e73f4f-5bde-4e1e-a82b-abc8ce2cb45b-image.png

                            After Restart tool stack
                            1990a307-c40c-440d-8b9b-d2171cc336a8-image.png

                            A 1 Reply Last reply Reply Quote 0
                            • A Offline
                              andriy.sultanov Vates 🪐 XAPI & Network Team @ph7
                              last edited by

                              @ph7 This update only covers the security issue described above. Fix for the stats issue will roll out later.

                              F P 2 Replies Last reply Reply Quote 3
                              • F Offline
                                flakpyro @andriy.sultanov
                                last edited by

                                @stormi installed on the same test machines i have the other batch of updates installed on. No issues after a reboot.

                                1 Reply Last reply Reply Quote 2
                                • P Offline
                                  ph7 @andriy.sultanov
                                  last edited by

                                  @andriy.sultanov
                                  Sorry, if I only could read...
                                  Anyhow, My updated host running on intel seems to work just fine.

                                  1 Reply Last reply Reply Quote 2
                                  • G Offline
                                    Greg_E @stormi
                                    last edited by

                                    @stormi

                                    My lab is down for a few days, so no testing for me. And it is AMD so maybe not useful. I probably won't be back running until Friday.

                                    stormiS 1 Reply Last reply Reply Quote 0
                                    • stormiS Offline
                                      stormi Vates 🪐 XCP-ng Team @Greg_E
                                      last edited by

                                      @Greg_E Thanks for letting me know. It is useful to make sure that it's still working on any kind of hardware, but your lab won't participate this time 🙂

                                      1 Reply Last reply Reply Quote 0
                                      • B Offline
                                        bufanda
                                        last edited by

                                        Installed the patches on my lab pool and both hosts are up and running and no issues so far.

                                        1 Reply Last reply Reply Quote 1
                                        • X Offline
                                          XCP-ng-JustGreat
                                          last edited by

                                          Latest urgent updates installed on 3-node Intel pool. Everything is working as before including the pre-production code "no stats" issue, but that still resolves following xe-toolstack-restart command. Since it is currently Microsoft patch Tuesday here, the latest Windows 11-24H2 2025-05 cumulative update was installed to VM along with various Linux VM updates and live VM host migrations. All working well including latest af03c Xen Orchestra from source (XOS).

                                          1 Reply Last reply Reply Quote 1
                                          • A Offline
                                            Andrew Top contributor @stormi
                                            last edited by

                                            @stormi Upgraded my test 8.3 hosts, several Intel and AMD Zen 3. So far, so good.

                                            1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post