XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Poor VM performance after migrating from VMWare to XCP-NG

    Scheduled Pinned Locked Moved Migrate to XCP-ng
    15 Posts 5 Posters 3.2k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DanpD Offline
      Danp Pro Support Team
      last edited by

      A few questions for you --

      • Are you booting in BIOS or UEFI mode? FWIW, I've read where others found the migrated VM ran better in UEFI mode.

      • How did you migrate the VM (XO's V2V tool vs manual export / import)?

      • Have you tried creating a brand new VM using the appropriate template and then attaching the existing disk in place of the default one?

      M 2 Replies Last reply Reply Quote 0
      • M Offline
        markds @Danp
        last edited by

        1. Booting via BIOS mode (both in VMWare and XCP-NG)

        2. Used XO's vmware import tool
          (from memory I believe the VM was powered off and then imported vs taking a snapshot then importing, but can not be sure)

        3. Have not tried creating a brand new VM and attaching the disk. Do you suspect the import has copied over some strange config impacting performance?

        BTW thanks for the prompt response @Danp

        DanpD R 2 Replies Last reply Reply Quote 0
        • DanpD Offline
          Danp Pro Support Team @markds
          last edited by

          @markds said in Poor VM performance after migrating from VMWare to XCP-NG:

          Have not tried creating a brand new VM and attaching the disk

          You should definitely try this. It was reported elsewhere on this forum that this improved performance, but I don't recall if this was for all VMs or specific to Windows.

          1 Reply Last reply Reply Quote 0
          • M Offline
            markds @Danp
            last edited by

            @Danp

            No luck... still

            • I created a full clone of the migrated VM
            • Ensured the clone was shutdown.
            • Create a new VM via the template (no disks, attached a cdrom, same mac as clone)
            • Edited the new VM
              - removed the reference the cdrom
              - attached the cloned disk using the uuid reference (which is a cool feature)
              - changed the boot order so that harddrive was first
            • ran our ip tables and the same issue 2mins 7 secs so I suspect the underlaying issue still exists...

            how do I confirm that the xen network drivers are in use:
            lsmod | grep xen
            xenfs 17639 1
            xen_netfront 22032 0
            xen_blkfront 17478 5

            does that not indicate that whilst xen_netfront module is loaded, no device is using that module?

            DanpD 1 Reply Last reply Reply Quote 0
            • DanpD Offline
              Danp Pro Support Team @markds
              last edited by

              @markds Looks that way. What output do you get for sudo dmesg | grep xen?

              M 3 Replies Last reply Reply Quote 0
              • M Offline
                markds @Danp
                last edited by

                @Danp

                [ 0.824078] xen/balloon: Initialising balloon driver.
                [ 0.825817] xen-balloon: Initialising balloon driver.
                [ 0.848060] Switching to clocksource xen
                [ 0.861416] xen: --> pirq=16 -> irq=8 (gsi=8)
                [ 0.861486] xen: --> pirq=17 -> irq=12 (gsi=12)
                [ 0.861531] xen: --> pirq=18 -> irq=1 (gsi=1)
                [ 0.861580] xen: --> pirq=19 -> irq=6 (gsi=6)
                [ 0.861632] xen: --> pirq=20 -> irq=4 (gsi=4)
                [ 0.861692] xen: --> pirq=21 -> irq=7 (gsi=7)
                [ 0.916200] xen: --> pirq=22 -> irq=23 (gsi=23)
                [ 1.135771] xen: --> pirq=23 -> irq=28 (gsi=28)
                [ 1.936447] vbd vbd-5696: 19 xenbus_dev_probe on device/vbd/5696

                1 Reply Last reply Reply Quote 0
                • M Offline
                  markds @Danp
                  last edited by

                  @Danp

                  Actually maybe its a red herring...

                  ethtool -i eth3
                  driver: vif
                  version:
                  firmware-version:
                  bus-info: vif-2

                  Also I tested using iperf:
                  vm <-> vm: 9.78 Gbits/sec (on the same xcp-ng host)
                  vm <-> exsi: 0.97 Gbits/sec (network is 1GB)

                  As a final test I ran:
                  rmmod xen_netfront
                  and as expected the vms lost network access

                  So maybe the issue is elsewhere

                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by olivierlambert

                    Can you reproduce the issue by creating a new Linux VM from scratch on XCP-ng? (ie installing the OS etc.) This will help to understand if it's a setup-wise or VM wise issue

                    M 1 Reply Last reply Reply Quote 0
                    • M Offline
                      markds @Danp
                      last edited by

                      @Danp
                      I should also correct my previous answer...

                      Looks like some of the dmesg are in a different case...

                      dmesg | grep -i xen
                      [ 0.000000] DMI: Xen HVM domU, BIOS 4.13 04/11/2024
                      [ 0.000000] Hypervisor detected: Xen HVM
                      [ 0.000000] Xen version 4.13.
                      [ 0.000000] Xen Platform PCI: I/O protocol version 1
                      [ 0.000000] Netfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated NICs.
                      [ 0.000000] Blkfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated disks.
                      [ 0.000000] ACPI: RSDP 0x00000000000EA020 00024 (v02 Xen)
                      [ 0.000000] ACPI: XSDT 0x00000000FC00A7C0 00044 (v01 Xen HVM 00000000 HVML 00000000)
                      [ 0.000000] ACPI: FACP 0x00000000FC00A370 000F4 (v04 Xen HVM 00000000 HVML 00000000)
                      [ 0.000000] ACPI: DSDT 0x00000000FC001040 092A3 (v02 Xen HVM 00000000 INTL 20160527)
                      [ 0.000000] ACPI: APIC 0x00000000FC00A470 00260 (v02 Xen HVM 00000000 HVML 00000000)
                      [ 0.000000] ACPI: HPET 0x00000000FC00A750 00038 (v01 Xen HVM 00000000 HVML 00000000)
                      [ 0.000000] ACPI: WAET 0x00000000FC00A790 00028 (v01 Xen HVM 00000000 HVML 00000000)
                      [ 0.000000] Booting paravirtualized kernel on Xen HVM
                      [ 0.000000] Xen HVM callback vector for event delivery is enabled
                      [ 0.113732] Xen: using vcpuop timer interface
                      [ 0.113741] installing Xen timer for CPU 0
                      [ 0.390521] installing Xen timer for CPU 1
                      [ 0.492037] installing Xen timer for CPU 2
                      [ 0.590329] installing Xen timer for CPU 3
                      [ 0.918283] xen/balloon: Initialising balloon driver.
                      [ 0.920100] xen-balloon: Initialising balloon driver.
                      [ 0.952068] Switching to clocksource xen
                      [ 0.967516] xen: --> pirq=16 -> irq=8 (gsi=8)
                      [ 0.967593] xen: --> pirq=17 -> irq=12 (gsi=12)
                      [ 0.967645] xen: --> pirq=18 -> irq=1 (gsi=1)
                      [ 0.967696] xen: --> pirq=19 -> irq=6 (gsi=6)
                      [ 0.967755] xen: --> pirq=20 -> irq=4 (gsi=4)
                      [ 0.967818] xen: --> pirq=21 -> irq=7 (gsi=7)
                      [ 1.025049] xen: --> pirq=22 -> irq=23 (gsi=23)
                      [ 1.228952] xen: --> pirq=23 -> irq=28 (gsi=28)
                      [ 1.571668] XENBUS: Device with no driver: device/vbd/5696
                      [ 1.573958] XENBUS: Device with no driver: device/vbd/768
                      [ 1.577690] XENBUS: Device with no driver: device/vif/0
                      [ 1.579893] XENBUS: Device with no driver: device/vif/1
                      [ 1.584074] XENBUS: Device with no driver: device/vif/2
                      [ 1.588143] XENBUS: Device with no driver: device/vif/3
                      [ 2.040651] vbd vbd-5696: 19 xenbus_dev_probe on device/vbd/5696
                      [ 2.152854] Initialising Xen virtual ethernet driver.

                      1 Reply Last reply Reply Quote 0
                      • M Offline
                        markds @olivierlambert
                        last edited by

                        @olivierlambert
                        So I copied the firewall rules script over to another linux vm (running the current debian version) and that work properly (finished in 5 seconds).

                        It would seem like all the vms that are not performing well are based on legacy versions of Debian.

                        I would have thought a kernel version of 3.2.102 would have been recent enough to get decent performance.

                        Do you know of any issues where Xen performs badly for older Debian versions?

                        1 Reply Last reply Reply Quote 0
                        • olivierlambertO Offline
                          olivierlambert Vates 🪐 Co-Founder CEO
                          last edited by

                          I would have thought a kernel version of 3.2.102 would have been recent enough to get decent performance.

                          😬 Kernel 3.2 was out in 2012 and EOL since 2018. That's very old and not secure (I doubt anyone would have backported any security patches since then). That's 6 years worth of CVEs.

                          I don't know if PV mode would be better. It's a Debian 32 bit or 64 bit?

                          1 Reply Last reply Reply Quote 0
                          • R Offline
                            rtjdamen @markds
                            last edited by

                            @markds we have seen major improvement when booting a vm in uefi, for some reason vms created on vmware with bios are terrible slow on xcp-ng, switching to uefi made them running normal again.

                            Tristis OrisT 1 Reply Last reply Reply Quote 0
                            • Tristis OrisT Offline
                              Tristis Oris Top contributor @rtjdamen
                              last edited by

                              @rtjdamen switch from bios to uefi is possible? VM can't startup after that.

                              R 1 Reply Last reply Reply Quote 0
                              • R Offline
                                rtjdamen @Tristis Oris
                                last edited by

                                @Tristis-Oris u need to change the boot records for this, on windows we use a specific tool for this that converts the MBR disk to a GPT disk, i am not shure how the proces work in linux but i know that can be done there as well.

                                1 Reply Last reply Reply Quote 0

                                Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                                Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                                With your input, this post could be even better 💗

                                Register Login
                                • First post
                                  Last post