XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Live Migration Very Slow

    Scheduled Pinned Locked Moved Management
    9 Posts 4 Posters 560 Views 5 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S Offline
      stevewest15
      last edited by stevewest15

      Xen Orchestra, commit 32228
      Master, commit 32228

      Hi,

      I have a two host pool that uses shared storage for all VMs. When I try to perform a VM migrate, using either the management network (1Gbs) or the 20Gbps NFS network, the live migration can take a long time (see example below):
      91eb5e25-fd80-460e-9462-7b1e2539108c-image.png
      d6df8e38-5ee1-48da-973b-1551546accf1-image.png

      I thought when the VM is using the shared storage of a pool, then moving VMs should be very fast as there is no disk data that needs to be moved other than the VM ram.

      Both hosts Control Domains have assigned 16 GB of memory.

      Source host shows the following:

      185a3135-4fcd-4d86-b44b-f80aaf515875-image.png

      71dc9539-98af-4662-bf70-bcec58c82d60-image.png

      240553e9-4e47-453a-90e1-7a8db02b9b61-image.png

      Target host shows the following:

      d1bd33bd-8fb4-4716-a259-cdbd93d5f9bb-image.png

      5e649d7b-30dd-436e-9bc8-b22a72da3e22-image.png

      68060277-1cb7-4a59-a184-9848c7ed9e42-image.png

      Any recommendations on what could be causing the slow live migration if the VM resides on the pool shared storage?

      Also, our XOCE VM resides on a different pool than the pool I'm performing the live migration. Wasn't sure if that matters or not.

      Thank You,

      SW

      1 Reply Last reply Reply Quote 0
      • andrewperryA Offline
        andrewperry
        last edited by

        Were you able to work it out? I have a 32GB RAM domU that keeps failing migration between nodes unfortunately!

        S 1 Reply Last reply Reply Quote 0
        • S Offline
          stevewest15 @andrewperry
          last edited by

          andrewperry I couldn't get the live migration to work on large VMs. To move large VMs, I ended up taking a snapshot of the VM and then creating a new VM from that snapshot. It's not ideal but had to do that as we needed to retire some old servers and I couldn't get the live migration to work w/ large VMs.

          andrewperryA 1 Reply Last reply Reply Quote 1
          • andrewperryA Offline
            andrewperry @stevewest15
            last edited by

            stevewest15 that is a shame! Thanks for sharing!

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Offline
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by

              Live migration speed per VM isn't really fast due how memory is split is very small 4k blocks. However, it scales OK if you migrate multiple VMs at once. Obviously, this is not a solution in your case where you need faster migration per VM. This is a topic we'd like to work on in the following months 🙂

              ForzaF 1 Reply Last reply Reply Quote 3
              • ForzaF Offline
                Forza @olivierlambert
                last edited by Forza

                Hi, sorry for revisiting an older topic, but we have the same issue with slow VM migration. We changed from 2x1G to 2x10G network, but the migration performance from one host's local SR to another host local SR is not much improved.

                9eb4913a-3ed6-4376-b653-23090e323bd6-image.png

                611ce830-d3cc-4153-89b9-9d4e1c92d6b4-image.png

                Using XCP-ng 8.2.1 with up-to-date patches. Local storage on both hosts is SSD RAID1, ext4.

                It would be very good if we could improve this situation. Currently we are seeing only 5% network utilisation.

                olivierlambertO andrewperryA 2 Replies Last reply Reply Quote 0
                • olivierlambertO Offline
                  olivierlambert Vates 🪐 Co-Founder CEO @Forza
                  last edited by

                  Forza Sorry but that's unrelated in this topic. OP talks about live migration with a shared SR. Storage migration is really another topic.

                  ForzaF 1 Reply Last reply Reply Quote 0
                  • ForzaF Offline
                    Forza @olivierlambert
                    last edited by

                    olivierlambert aha, I misinderstood. Should I open another topic or perhaps a support ticket?

                    1 Reply Last reply Reply Quote 0
                    • andrewperryA Offline
                      andrewperry @Forza
                      last edited by andrewperry

                      Forza There are various topics touching on how SMAPIv1 is a bottleneck here, eg: https://xcp-ng.org/forum/topic/9389/backup-migration-performance/8 - that's probably not the best example as it is also more on backups than migration!

                      1 Reply Last reply Reply Quote 0
                      • First post
                        Last post