XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    [RHEL kernel bug] XCP vm fails to boot after newest kernel applied.

    Scheduled Pinned Locked Moved Solved Compute
    49 Posts 12 Posters 3.0k Views 14 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • G Offline
      Greg_E @bberndt
      last edited by

      @bberndt

      Is there something different about Alma? Friday I may have some time to fiddle and can try a Rocky 8 to see what happens.

      Is this an 8.2 and 8.3 issue or just 8.2? I have 8.2 in production and could try there, 8.3 in my lab with a very fresh build.

      B 1 Reply Last reply Reply Quote 0
      • B Offline
        bberndt @Greg_E
        last edited by

        @Greg_E said in XCP vm fails to boot after newest kernel applied.:

        @bberndt

        Is there something different about Alma? Friday I may have some time to fiddle and can try a Rocky 8 to see what happens.

        Is this an 8.2 and 8.3 issue or just 8.2? I have 8.2 in production and could try there, 8.3 in my lab with a very fresh build.

        Migrated to a XCP-ng 8.3 host. Xeon E5-2689 v4
        No change.

        G 1 Reply Last reply Reply Quote 0
        • G Offline
          Greg_E @bberndt
          last edited by

          @bberndt

          If I find an hour, I'll give Rocky 8 a try.

          Could you use LEAPP to migrate that to Alma, maybe they are doing something differently which is why mine are working.

          I can tell you, there is nothing special that I'm doing, my systems are as vanilla as they get.

          B 1 Reply Last reply Reply Quote 0
          • B Offline
            bberndt @Greg_E
            last edited by bberndt

            @Greg_E said in XCP vm fails to boot after newest kernel applied.:

            @bberndt

            If I find an hour, I'll give Rocky 8 a try.

            Could you use LEAPP to migrate that to Alma, maybe they are doing something differently which is why mine are working.

            I can tell you, there is nothing special that I'm doing, my systems are as vanilla as they get.

            @Greg_E
            google AI says what I think is familiar:
            Rocky Linux strives for 1:1 bug-for-bug compatibility with RHEL, while Alma Linux is more of an RHEL rebuild, making some adjustments and adding its own features

            G 1 Reply Last reply Reply Quote 0
            • G Offline
              Greg_E @bberndt
              last edited by

              @bberndt

              Ok, that might explain the difference.

              Would a LEAPP from Rocky 8 up to Alma 9 be possible and solve the issue?

              B 1 Reply Last reply Reply Quote 0
              • B Offline
                bufanda @Greg_E
                last edited by bufanda

                @Greg_E said in XCP vm fails to boot after newest kernel applied.:

                @bberndt
                I just installed a fresh Alma 8, did yum update to see what would happen, and it's still working. Gave out the same kernel as above.

                Alma8.png

                This was installed UEFI on XCP-ng 8.3 which was a fresh install a few days ago from a nightly (near release?) ISO. It was installed to an NFS share, 2 cores and 4GB with an Intel i1000 interface. Xenserver tools 8.4.0-1 installed.

                There are no extra packages installed yet, could this be a package conflict.

                Anything else I can check to see why mine works and others are failing?

                Hmm....I did the same and that VM died just like any other with the broken kernel.
                Although I used the Cloud Image so not completly new install from scratch.
                Maybe I'll check building a new cloud image what will happen then.

                G 1 Reply Last reply Reply Quote 0
                • G Offline
                  Greg_E @bufanda
                  last edited by

                  @bufanda

                  How many of these that are failing have been upgraded from a previous version? Could it be something left over from EL7 or early EL8?

                  My Alma 8.10 base install is still running fine, did a yum update to apply a few more things and reboot and still working with the same kernel version above. But again, this was a clean fresh install, not something that's been running for a while.

                  B 1 Reply Last reply Reply Quote 0
                  • anthonyperA Offline
                    anthonyper Xen Guru
                    last edited by

                    FYI, there's a patch submitted to linux-stable (6.6 and earlier) but not yet in a stable release:
                    https://lore.kernel.org/stable/20250411160833.12944-1-jason.andryuk@amd.com/

                    I guess we'll have to wait until this is picked up by Linux, then Red Hat will have to pick that as well.

                    1 Reply Last reply Reply Quote 0
                    • B Offline
                      bberndt @Greg_E
                      last edited by

                      @Greg_E said in XCP vm fails to boot after newest kernel applied.:

                      @bberndt

                      Ok, that might explain the difference.

                      Would a LEAPP from Rocky 8 up to Alma 9 be possible and solve the issue?

                      I did a (not LEAPP, but a migration script from Alama) from Rocky 8 to Alma 8, and it died. None of my Rocky 9's have had a problem so far, and of course end up with a completely different kernel.

                      B 1 Reply Last reply Reply Quote 0
                      • B Offline
                        bufanda @Greg_E
                        last edited by bufanda

                        @Greg_E They are all running for a while but none where upgrades from RHEL 7 to RHEL 8. I don't do LEAPPs that gives me more often errors than it works. What I did though on one is replacing the System VDI with a newer one, but that was Alma 8 to Alma 9 and that one has no issues.

                        1 Reply Last reply Reply Quote 0
                        • B Offline
                          bufanda @bberndt
                          last edited by

                          @bberndt you Rocky/Alma 9 will be fine since it's a bug in the kernel for 8 only.

                          1 Reply Last reply Reply Quote 1
                          • X Offline
                            XCP-ng-JustGreat
                            last edited by

                            This is just a "me too" reply to indicate that I am also experiencing the boot failure immediately after upgrading AlmaLinux 8.10 to the latest kernel 4.18.0-553.50.1. Hopefully, the kernel fix will get integrated soon into the affected and popular RedHat 8 derivatives so that Alma and Rocky 8 et al. can continue to run on our favorite hypervisor. This is a bad one.

                            G 1 Reply Last reply Reply Quote 0
                            • G Offline
                              Greg_E @XCP-ng-JustGreat
                              last edited by

                              @XCP-ng-JustGreat

                              Strange that my fresh Alma 8.10 is still working after updates, makes me think k there is more to this than just the kernel.

                              1 Reply Last reply Reply Quote 0
                              • X Offline
                                XCP-ng-JustGreat
                                last edited by

                                @Greg_E Wondering what might be the difference? I am using the XenServer Linux tools version 8.4.0-1 and BIOS boot firmware.

                                G 1 Reply Last reply Reply Quote 0
                                • G Offline
                                  Greg_E @XCP-ng-JustGreat
                                  last edited by

                                  @XCP-ng-JustGreat

                                  I'm running the same guest tools.

                                  Could it be a processor related issue? I'm running AMD v1756b in my lab, haven't had time to try in production that is Intel Silver.

                                  B 1 Reply Last reply Reply Quote 0
                                  • X Offline
                                    XCP-ng-JustGreat
                                    last edited by XCP-ng-JustGreat

                                    @Greg_E Maybe it only affects Intel? That would be a new wrinkle. Perhaps other AMD users can confirm if the latest AlmaLinux 8 kernel runs fine on their AMD XCP-ng 8.3 hosts? Just read the above referenced RedHat bug report here: https://access.redhat.com/solutions/7116307 Sure enough, it says Xen + Intel in problem description. Check it out.

                                    1 Reply Last reply Reply Quote 0
                                    • B Offline
                                      bufanda @Greg_E
                                      last edited by

                                      @Greg_E according to the Knowledge base of Redhawks it is den Witz Intel CPUs. So you AMD isn‘t affected

                                      1 Reply Last reply Reply Quote 0
                                      • P Offline
                                        phil
                                        last edited by

                                        According to https://access.redhat.com/solutions/7116307 the bug is solved, but I don't get newer Kernels on Rocky yet.

                                        G 1 Reply Last reply Reply Quote 0
                                        • G Offline
                                          Greg_E @phil
                                          last edited by

                                          @phil

                                          Sad that even this documentation is locked behind a Redhat account, I can only read part of the problem description. Not going to make an account because it really doesn't affect me right now.

                                          B 1 Reply Last reply Reply Quote 0
                                          • B Offline
                                            bberndt @Greg_E
                                            last edited by

                                            @Greg_E
                                            I made a free account.
                                            Resolution
                                            The issue has been resolved with the errata: RHBA-2025:4337. Hence, update the kernel to kernel-4.18.0-553.51.1.el8_10 to fix the issue.

                                            id assume it arrives soon? I as well, haven't seen it yet.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post