XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Guest running kernel 6.8 hangs after a while

    Scheduled Pinned Locked Moved Solved XCP-ng
    17 Posts 7 Posters 1.8k Views 7 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • apzA Offline
      apz
      last edited by

      While running a test virtual for the upcoming Ubuntu 24.04, I noticed once it went from 6.6 to 6.8 kernel it started flooding the kernel with messages like "BUG: Bad page state in process" about virtually any process that was active at the moment. After a while the guest just hangs.

      This is not an Ubuntu specific bug, but it's probably going to be pretty noticeable under Ubuntu and its spinoffs once the new release comes out.

      I've created a bug against the kernel package here:

      https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2056706

      After some testing it would appear to be something that gets triggered when the guest is running under Xen. For those who have an account there, could you verify this issue happens on your machine (even with just a barebones install updated to 6.8) and then hit "this affects me too" link on the bug report.

      1 Reply Last reply Reply Quote 0
      • Q Offline
        qaguya
        last edited by

        today 6.8.0-31-generic kernel problem fixed.

        1 Reply Last reply Reply Quote 2
        • olivierlambertO Offline
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          Hi,

          Any specific edition of Ubuntu, I mean a desktop or server one?

          apzA 1 Reply Last reply Reply Quote 0
          • apzA Offline
            apz @olivierlambert
            last edited by

            @olivierlambert Doesn't seem to matter, I tried various installers of the 24.04 beta, they all develop the issue with 6.8 kernel which they get after the first upgrade. I first encountered this with a headless homelab virtual.

            I just brought this up here as 24.04 will be out next month and after that there will most likely be a lot of people reacting to the issue. It appears to be related running as a guest under Xen. I couldn't replicate the issue with real hardware or KVM.

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by

              Thanks, I can reproduce with the nightly Ubuntu build. Asked internally so someone take a look. I suspect it will be spotted (or maybe already) since AWS still using Xen at scale (it would be VERY visible)

              apzA 1 Reply Last reply Reply Quote 0
              • apzA Offline
                apz @olivierlambert
                last edited by

                @olivierlambert This appears to fix it:

                https://lore.kernel.org/all/171154167446.2671062.9127105384591237363.stgit@firesoul/

                1 Reply Last reply Reply Quote 2
                • olivierlambertO Offline
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  Great news!

                  R 1 Reply Last reply Reply Quote 0
                  • R Offline
                    redakula @olivierlambert
                    last edited by

                    @olivierlambert

                    Update now that 24.04 release date has been announced for the 25th of april.

                    The kernel has not been updated in the recently released beta and the bug is still present.

                    According to launcpad it is not even assigned so unsure of how much important Canonical considers it...
                    https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2056706

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Offline
                      olivierlambert Vates 🪐 Co-Founder CEO
                      last edited by olivierlambert

                      I'm not sure there's anything we can do, any idea @stormi or @yann ?

                      apzA 1 Reply Last reply Reply Quote 0
                      • apzA Offline
                        apz @olivierlambert
                        last edited by

                        @olivierlambert They operate with squeaky wheel mentality, the more "I have this too" presses that report gets, the likely it'll be someone eventually notices.

                        1 Reply Last reply Reply Quote 0
                        • olivierlambertO Offline
                          olivierlambert Vates 🪐 Co-Founder CEO
                          last edited by

                          @yann can you take a look at this and push Canonical on this?

                          yannY 1 Reply Last reply Reply Quote 0
                          • yannY Offline
                            yann Vates 🪐 XCP-ng Team
                            last edited by yann

                            Catching up with the subject...

                            The good news first: the patch is already in the 6.8 stable branch (as c8b7b2f158d9d4fb89cd2f68244af154f7549bb4), and part of v6.8.5.

                            The Ubuntu situation is:

                            • master kernel branch OTOH has been at v6.8.4 for 12 days, with only minor packaging adjustments since then
                            • latest tag on master is 6.8.0-25 (only used for lowlatency kernel as of today), based on v6.8.1
                            • they also have a 6.8.0-26 and changelog entries for -27 on master-next (all about s390)
                            • standard kernel at 6.8.0-22, based on v6.8.1 too

                            I'm not familiar with their internal processes, but I suspect they're probably in a validation cycle for upcoming upcoming version right now, and any such fix would have to wait for next one.

                            The tracker ticket for 6.8.0-27 seems to imply -27 packaging is "in progress". Still digging...

                            1 Reply Last reply Reply Quote 1
                            • yannY Offline
                              yann Vates 🪐 XCP-ng Team @olivierlambert
                              last edited by yann

                              Raised this on IRC, resulting in:

                              I'll see what I can do, we may still have some time to include this before the release

                              I guess that's pretty much all we can do on our side (and that's nice from them 😉 ).

                              1 Reply Last reply Reply Quote 1
                              • Q Offline
                                qaguya
                                last edited by

                                today 6.8.0-31-generic kernel problem fixed.

                                1 Reply Last reply Reply Quote 2
                                • olivierlambertO Offline
                                  olivierlambert Vates 🪐 Co-Founder CEO
                                  last edited by

                                  Yay \o/

                                  1 Reply Last reply Reply Quote 0
                                  • olivierlambertO olivierlambert marked this topic as a question on
                                  • olivierlambertO olivierlambert has marked this topic as solved on
                                  • olivierlambertO Offline
                                    olivierlambert Vates 🪐 Co-Founder CEO
                                    last edited by

                                    Thanks everyone here for the feedback!

                                    1 Reply Last reply Reply Quote 0
                                    • B Offline
                                      bvitnik
                                      last edited by bvitnik

                                      Has anyone been able to install Ubuntu 24.04 in VM from current official ISO? It seems that official ISOs (i.e. installer) still use unpatched kernel 6.8.0-22. Are there any newer ISO builds that I'm not aware of?

                                      EDIT:
                                      Sorry. False alarm. I screwed up my PXE settings. There was some leftover kernel and initrd images from beta versions of ISO. Kernel and initrd from latest ISOs work properly.

                                      1 Reply Last reply Reply Quote 1
                                      • T Offline
                                        Tweet3049
                                        last edited by Tweet3049

                                        I believe Proxmox Backup Server kernel Linux pbs 6.8.4-2-pve has also the same issue.
                                        updating to Linux pbs 6.8.12-1-pve solves.

                                        1 Reply Last reply Reply Quote 1
                                        • First post
                                          Last post