XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Nested Virtualization in xcp-ng

    Scheduled Pinned Locked Moved Compute
    10 Posts 3 Posters 88 Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • AlexanderKA Offline
      AlexanderK
      last edited by

      Right now we are migrating many servers from esxi to xcp-ng.
      Many users have vms with nested virtualization enabled. Some for docker desktop some other for android studio etc.
      They keep asking for this and i am searching for a solution.
      Do you have any plans to implement this? Any solution to propose?
      Thanks in advance

      1 Reply Last reply Reply Quote 0
      • A Offline
        abudef
        last edited by

        Nested virtualization doesn’t work very well in Xen. For example, when I set up a small test playground, I had XCP-ng 8.3 as the primary host and another XCP-ng running on it as a nested host. When I then booted Debian 12 on the nested host, it caused the entire nested host to crash and reboot. On the other hand, Windows VMs on the nested host run quite well.

        Later, when I was preparing a test lab, I installed ESXi 8.0U3e on a Dell server and then deployed four virtualized XCP-ng 8.3 hosts on top of it. A number of Linux and Windows VMs run on them without any issues.

        M 1 Reply Last reply Reply Quote 0
        • A Offline
          abudef
          last edited by abudef

          You can enable it in the VM configuration simply by clicking a button in Xen Orchestra:

          7fc30f0d-3c2d-487c-a0f1-d235701a3762-image.jpeg

          or use this command:
          xe vm-param-set platform:nested_virt=true uuid=<VM UUID> mistake here, see below

          AlexanderKA 1 Reply Last reply Reply Quote 0
          • M Offline
            MajorP93 @abudef
            last edited by

            @abudef said:

            Nested virtualization doesn’t work very well in Xen. For example, when I set up a small test playground, I had XCP-ng 8.3 as the primary host and another XCP-ng running on it as a nested host. When I then booted Debian 12 on the nested host, it caused the entire nested host to crash and reboot. On the other hand, Windows VMs on the nested host run quite well.

            Later, when I was preparing a test lab, I installed ESXi 8.0U3e on a Dell server and then deployed four virtualized XCP-ng 8.3 hosts on top of it. A number of Linux and Windows VMs run on them without any issues.

            Did you follow the official documentation for nested virtualization?

            https://docs.xcp-ng.org/guides/xcpng-in-a-vm/#nested-xcp-ng-using-xcp-ng

            Most importantly setting via command line on pool master:

            xe vm-param-set uuid=<UUID> platform:exp-nested-hvm=true

            and

            xe vm-param-set uuid=<UUID> platform:nic_type="e1000"
            A 1 Reply Last reply Reply Quote 0
            • A Offline
              abudef @MajorP93
              last edited by

              @MajorP93 Sure I did

              M 1 Reply Last reply Reply Quote 0
              • M Offline
                MajorP93 @abudef
                last edited by

                @abudef said:

                @MajorP93 Sure I did

                Hmm you previously mentioned different command so I was not sure if you really followed documentation correctly.

                A 1 Reply Last reply Reply Quote 0
                • A Offline
                  abudef @MajorP93
                  last edited by

                  @MajorP93 Overall it works, Windows VMs run more or less reliably, but I wasn’t able to resolve the nested host crashing when starting Debian. That’s why I put the free version of ESXi underneath the nested XCP-ng hosts, and that works reliably.

                  1 Reply Last reply Reply Quote 1
                  • AlexanderKA Offline
                    AlexanderK @abudef
                    last edited by

                    @abudef this causes some vms not to boot

                    A 1 Reply Last reply Reply Quote 0
                    • A Offline
                      abudef @AlexanderK
                      last edited by

                      @AlexanderK said:

                      @abudef this causes some vms not to boot

                      You mean that when you enable Nested virtualization in the VM settings under the Advanced tab, the VM then fails to boot?

                      AlexanderKA 1 Reply Last reply Reply Quote 0
                      • AlexanderKA Offline
                        AlexanderK @abudef
                        last edited by

                        @abudef exactly

                        1 Reply Last reply Reply Quote 0

                        Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                        Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                        With your input, this post could be even better 💗

                        Register Login
                        • First post
                          Last post