XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    leaf-coalesce: EXCEPTION. " Unexpected bump in size"

    Scheduled Pinned Locked Moved Compute
    13 Posts 4 Posters 3.1k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      topsecret @ronan-a
      last edited by topsecret

      @ronan-a This cluster based on Huawei CH121 V5 servers with Intel(R) Xeon(R) Gold 5120T CPU @ 2.20GHz, 320 Gb RAM and Intel(R) Xeon(R) Gold 6138T CPU @ 2.00GHz, 512 Gb RAM. We have more than a half free RAM and CPU on each server according to Xen Orchestra. SAS storage free capacity is about 45% (17Tb).
      Problem disks capacity are about 2Tb
      We can't stop virtual machines, it affects productive service

      1 Reply Last reply
      Reply Quote 0
      • T
        topsecret
        last edited by

        I powered off one VM with two 2Tb disks, overall coalesce time was about 3-4 hours.

        Running VM still has VDI to coalesce. I found proccess "/usr/bin/vhd-util coalesce --debug -n /dev/VG_XenStorage-de024eb7-ce14-5487-e229-7ca321b103a2/VHD-b5d6ab41-50dc-4116-a23c-e453b93ce161"
        Can I run it again to parallel coalesce process?

        O 1 Reply Last reply
        Reply Quote 0
        • O
          O_V_K @topsecret
          last edited by

          Hello!

          Same error " Unexpected bump in size" on different servers with xcp-ng 8.2.0.
          Hardware RAID5 and RAID10 used, with 8 SDD DC500M-DC600M. Only power off VM, Rescan, and wait about 8-10 minutes helps. Any solution or any updates can solve the problem?

          Thank you!

          1 Reply Last reply
          Reply Quote 0
          • olivierlambertO
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by olivierlambert

            Hi,

            Your SR is probably coalescing slower than you are adding data to your disk in live, and can't catch up.

            You might try to use CBT-enabled backup with XO to reduce the snapshot size.

            O 1 Reply Last reply
            Reply Quote 0
            • O
              O_V_K @olivierlambert
              last edited by

              Thank you!

              But, disk IO operations is very low during coalescing. All users are logged off from the server.

              1 Reply Last reply
              Reply Quote 0
              • olivierlambertO
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                So I can only suppose it's a Windows guest? Those guest are always writing a non-negligible quantity, and if your coalesce speed is slower than this, then, the coalesce process will detect data has grown faster than it merged, and it will fail.

                There's another possibility, to modify some coalesce timing to be more aggressive, that might solve it on your end.

                Following an old feedback on Github, you can try those values: https://github.com/xcp-ng/xcp/issues/298#issuecomment-557805054

                O 1 Reply Last reply
                Reply Quote 0
                • O
                  O_V_K @olivierlambert
                  last edited by

                  Thank you!

                  Yes, Windows VMs with guest tools installed.

                  1 Reply Last reply
                  Reply Quote 0
                  • olivierlambertO
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by

                    Keep us posted on the result 🙂

                    O 1 Reply Last reply
                    Reply Quote 0
                    • O
                      O_V_K @olivierlambert
                      last edited by

                      Dependig on hardware, any xcp-ng 8.2.0 host must be modified, if it running Windows VMs? My xcp-ng 8.2.0 host servers has powerful disk system, based on SSD and hardware RAID controller with onboard cache.

                      Thank you!

                      1 Reply Last reply
                      Reply Quote 0
                      • olivierlambertO
                        olivierlambert Vates 🪐 Co-Founder CEO
                        last edited by

                        No, it really depends on many factors. There's no universal tuning.

                        1 Reply Last reply
                        Reply Quote 0

                        Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                        Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                        With your input, this post could be even better 💗

                        Register Login
                        • First post
                          Last post