XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    leaf-coalesce: EXCEPTION. " Unexpected bump in size"

    Scheduled Pinned Locked Moved Compute
    13 Posts 4 Posters 1.1k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ronan-aR Offline
      ronan-a Vates 🪐 XCP-ng Team @olivierlambert
      last edited by

      @olivierlambert Of course. It's a new time related to this issue: https://xcp-ng.org/forum/topic/6732/vdi-could-not-be-coalesced/7?_=1692200431466

      @topsecret What's your host configuration? CPU and drive please? An overused CPU can be the cause of the problem, a slow disk, too many writes in VMs... Many factors can play a role.

      T 1 Reply Last reply Reply Quote 1
      • T Offline
        topsecret @ronan-a
        last edited by topsecret

        @ronan-a This cluster based on Huawei CH121 V5 servers with Intel(R) Xeon(R) Gold 5120T CPU @ 2.20GHz, 320 Gb RAM and Intel(R) Xeon(R) Gold 6138T CPU @ 2.00GHz, 512 Gb RAM. We have more than a half free RAM and CPU on each server according to Xen Orchestra. SAS storage free capacity is about 45% (17Tb).
        Problem disks capacity are about 2Tb
        We can't stop virtual machines, it affects productive service

        1 Reply Last reply Reply Quote 0
        • T Offline
          topsecret
          last edited by

          I powered off one VM with two 2Tb disks, overall coalesce time was about 3-4 hours.

          Running VM still has VDI to coalesce. I found proccess "/usr/bin/vhd-util coalesce --debug -n /dev/VG_XenStorage-de024eb7-ce14-5487-e229-7ca321b103a2/VHD-b5d6ab41-50dc-4116-a23c-e453b93ce161"
          Can I run it again to parallel coalesce process?

          O 1 Reply Last reply Reply Quote 0
          • O Offline
            O_V_K @topsecret
            last edited by

            Hello!

            Same error " Unexpected bump in size" on different servers with xcp-ng 8.2.0.
            Hardware RAID5 and RAID10 used, with 8 SDD DC500M-DC600M. Only power off VM, Rescan, and wait about 8-10 minutes helps. Any solution or any updates can solve the problem?

            Thank you!

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by olivierlambert

              Hi,

              Your SR is probably coalescing slower than you are adding data to your disk in live, and can't catch up.

              You might try to use CBT-enabled backup with XO to reduce the snapshot size.

              O 1 Reply Last reply Reply Quote 0
              • O Offline
                O_V_K @olivierlambert
                last edited by

                Thank you!

                But, disk IO operations is very low during coalescing. All users are logged off from the server.

                1 Reply Last reply Reply Quote 0
                • olivierlambertO Offline
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  So I can only suppose it's a Windows guest? Those guest are always writing a non-negligible quantity, and if your coalesce speed is slower than this, then, the coalesce process will detect data has grown faster than it merged, and it will fail.

                  There's another possibility, to modify some coalesce timing to be more aggressive, that might solve it on your end.

                  Following an old feedback on Github, you can try those values: https://github.com/xcp-ng/xcp/issues/298#issuecomment-557805054

                  stormi created this issue in xcp-ng/xcp

                  closed XCP-ng 8.0 / CH 8.0 coalesce issues #298

                  O 1 Reply Last reply Reply Quote 0
                  • O Offline
                    O_V_K @olivierlambert
                    last edited by

                    Thank you!

                    Yes, Windows VMs with guest tools installed.

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Offline
                      olivierlambert Vates 🪐 Co-Founder CEO
                      last edited by

                      Keep us posted on the result 🙂

                      O 1 Reply Last reply Reply Quote 0
                      • O Offline
                        O_V_K @olivierlambert
                        last edited by

                        Dependig on hardware, any xcp-ng 8.2.0 host must be modified, if it running Windows VMs? My xcp-ng 8.2.0 host servers has powerful disk system, based on SSD and hardware RAID controller with onboard cache.

                        Thank you!

                        1 Reply Last reply Reply Quote 0
                        • olivierlambertO Offline
                          olivierlambert Vates 🪐 Co-Founder CEO
                          last edited by

                          No, it really depends on many factors. There's no universal tuning.

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post