XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Memory in vm half as fast after migration of vm.

    Scheduled Pinned Locked Moved Compute
    41 Posts 6 Posters 15.8k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • olivierlambertO Online
      olivierlambert Vates 🪐 Co-Founder CEO
      last edited by

      You should probably start to compare XAPI record for the VM, before and after the migration, in case anything changed.

      A 1 Reply Last reply Reply Quote 0
      • A Offline
        Andreas @olivierlambert
        last edited by

        @olivierlambert XAPI records this is a bit beyond my knowledge.
        I tried to make a brand new VM Ubuntu 20.04 server
        just to see so it was not the virtual machine it was wrong in but
        unfortunately the same result again.

        1 Reply Last reply Reply Quote 0
        • olivierlambertO Online
          olivierlambert Vates 🪐 Co-Founder CEO
          last edited by

          xe vm-param-list uuid=<VM UUID>. Compare the output before and after migration.

          A 1 Reply Last reply Reply Quote 0
          • A Offline
            Andreas @olivierlambert
            last edited by

            @olivierlambert
            Okay find a value that is different
            the test is done on the new VM which has 6 GIB in Ram
            This is the value before migration
            memory-target ( RO): 0
            after migration
            memory-target ( RO): 6442450944
            and after reboot of VM
            memory-target ( RO): 0

            1 Reply Last reply Reply Quote 0
            • olivierlambertO Online
              olivierlambert Vates 🪐 Co-Founder CEO
              last edited by olivierlambert

              That's weird, despite the fact your VM is set to static, right? (same dynamic min, max and static max?)

              edit: is it the only diff?

              A 1 Reply Last reply Reply Quote 0
              • A Offline
                Andreas @olivierlambert
                last edited by Andreas

                @olivierlambert
                no this change from this
                memory-actual ( RO): 6442455040
                to this
                memory-actual ( RO): 6442450944

                and its below but they probably have no significance
                start-time
                console-uuids
                dom-id
                VCPUs-utilisation
                guest-metrics-last-updated
                b2471d1e-2d28-44c6-af0e-c555e9ecf100-image.png

                1 Reply Last reply Reply Quote 0
                • olivierlambertO Online
                  olivierlambert Vates 🪐 Co-Founder CEO
                  last edited by

                  That's weird. We'll see if we can reproduce this. @Darkbeldin will try when he can (probably in January)

                  A 1 Reply Last reply Reply Quote 0
                  • A Offline
                    Andreas @olivierlambert
                    last edited by

                    @olivierlambert
                    Okay thanks
                    and happy New Year 🙂

                    1 Reply Last reply Reply Quote 0
                    • olivierlambertO Online
                      olivierlambert Vates 🪐 Co-Founder CEO
                      last edited by

                      You too!

                      A 1 Reply Last reply Reply Quote 0
                      • A Offline
                        Andreas @olivierlambert
                        last edited by

                        @olivierlambert
                        Sorry to disturb you.
                        Okay just to verify that there was nothing wrong with the physical servers.
                        So I took 2 identical PCs and installed clean new xcp-ng 8.2
                        then install a virtual machine with static 4GB of memory and with guest tools.
                        Install redis and ran the test
                        then migrated VM to other pc and ran the test and the speed was half.
                        Took out the result before and after attached the files.

                        1.Before migration.txt
                        2.After migration.txt

                        DarkbeldinD 1 Reply Last reply Reply Quote 0
                        • DarkbeldinD Offline
                          Darkbeldin Vates 🪐 Pro Support Team @Andreas
                          last edited by

                          @andreas Hi Andreas,

                          After testing it on my side i can confirm i reproduce the issue.
                          I will discuss it at dev level and get back to you.

                          A ForzaF 2 Replies Last reply Reply Quote 2
                          • A Offline
                            Andreas @Darkbeldin
                            last edited by

                            @darkbeldin
                            Okay Thanks

                            DarkbeldinD 1 Reply Last reply Reply Quote 0
                            • ForzaF Offline
                              Forza @Darkbeldin
                              last edited by

                              @darkbeldin said in Memory in vm half as fast after migration of vm.:

                              @andreas Hi Andreas,

                              After testing it on my side i can confirm i reproduce the issue.
                              I will discuss it at dev level and get back to you.

                              This seems quite an important find. Please let is know how this goes.

                              1 Reply Last reply Reply Quote 0
                              • DarkbeldinD Offline
                                Darkbeldin Vates 🪐 Pro Support Team @Andreas
                                last edited by Darkbeldin

                                @andreas

                                So I was doing some testing before reporting to dev team and I have a behavior I will like you to check if you reproduce:
                                my clean VM report like this

                                yachy@ubuntuyachy:~$ redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q
                                SET: 156152.41 requests per second
                                GET: 168180.28 requests per second
                                LPUSH: 156421.08 requests per second
                                LPOP: 159757.17 requests per second
                                

                                That's my reference, when I migrate to another host it report like this:

                                yachy@ubuntuyachy:~$ redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q
                                SET: 55718.07 requests per second
                                GET: 58683.72 requests per second
                                LPUSH: 55742.91 requests per second
                                LPOP: 54775.01 requests per second
                                

                                If I reboot it goes back to original reporting but if I migrate back to the original host without rebooting it report like that.

                                redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q
                                SET: 138092.94 requests per second
                                GET: 153151.08 requests per second
                                LPUSH: 147004.78 requests per second
                                LPOP: 148115.23 requests per second
                                

                                So not perfect as reference but way better than after migration.
                                As I want to be thorough before reporting could you check if you reproduce that?
                                So:

                                • migrate to another host
                                • make the test
                                • migrate back to the original host
                                • make the test

                                Thanks for your help.

                                A 1 Reply Last reply Reply Quote 0
                                • A Offline
                                  Andreas @Darkbeldin
                                  last edited by

                                  @darkbeldin
                                  Hello
                                  I installed clean new xcp-ng 8.2 on 2 identical PCs name host1 and host2 then updated to latest "yum update"
                                  then install a virtual machine ubuntu 20.04 with static 4GB of memory and with guest tools.
                                  Install redis-server
                                  Then I did the test
                                  on host1
                                  root@ramtest:/home/andreas# redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q
                                  SET: 243368.20 requests per second
                                  GET: 261917.23 requests per second
                                  LPUSH: 257499.67 requests per second
                                  LPOP: 264830.50 requests per second

                                  Then migrate to host2 got lower speed
                                  root@ramtest:/home/andreas# redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q
                                  SET: 92055.60 requests per second
                                  GET: 95297.09 requests per second
                                  LPUSH: 95570.31 requests per second
                                  LPOP: 95401.64 requests per second

                                  Then back to host1 got almost the same speed
                                  root@ramtest:/home/andreas# redis-benchmark -r 1000000 -n 2000000 -t get,set,lpush,lpop -P 16 -q
                                  SET: 238010.23 requests per second
                                  GET: 253100.48 requests per second
                                  LPUSH: 259100.92 requests per second
                                  LPOP: 259134.50 requests per second

                                  DarkbeldinD 1 Reply Last reply Reply Quote 0
                                  • DarkbeldinD Offline
                                    Darkbeldin Vates 🪐 Pro Support Team @Andreas
                                    last edited by

                                    @andreas Ok so migrating back to the original host give us a small perf issue but clearly not what we see when we migrate to another host.
                                    I will report it like that thanks for the test Andreas 😉

                                    A 1 Reply Last reply Reply Quote 0
                                    • A Offline
                                      Andreas @Darkbeldin
                                      last edited by

                                      @darkbeldin
                                      Okay
                                      Did more tests
                                      Started on host1 normal speed
                                      migrate to host2
                                      make the test got lower speed
                                      restart vm
                                      make the test on host2
                                      Got normal speed
                                      migrate to host1
                                      make the test got lower speed
                                      migrate to host2 normal speed

                                      so it seems to be something that happens after first migrating to another host

                                      I have a third exactly the same pc i should test install on it
                                      and see what happens if i move vm to host3 after moving to host2
                                      but I have to do it tomorrow, I do not have time now.

                                      DarkbeldinD 2 Replies Last reply Reply Quote 0
                                      • DarkbeldinD Offline
                                        Darkbeldin Vates 🪐 Pro Support Team @Andreas
                                        last edited by

                                        @andreas Yes i tested it no need to do it, migrating to a third hosts result to half perf has first migration.

                                        1 Reply Last reply Reply Quote 0
                                        • DarkbeldinD Offline
                                          Darkbeldin Vates 🪐 Pro Support Team @Andreas
                                          last edited by

                                          @andreas Ok so after discussing it with Dev team the issue has been identified.
                                          The trouble is linked to TSC management in the VM.
                                          You can work around the issue by setting the VM:

                                          xe vm-param-set uuid=<VM_UUID> platform:tsc_mode=2
                                          

                                          But be aware we can not recommend this settings to go to a production VM.
                                          TSC clock won't be emulated at all if you enable this settings. So you might have some weird time behavior during migration.

                                          A TheNorthernLightT A 3 Replies Last reply Reply Quote 0
                                          • A Offline
                                            Andreas @Darkbeldin
                                            last edited by

                                            @darkbeldin Okay thanks
                                            I did test this and it worked.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post