XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Nested Virtualization in xcp-ng

    Scheduled Pinned Locked Moved Compute
    11 Posts 4 Posters 119 Views 5 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A Offline
      abudef
      last edited by

      Nested virtualization doesn’t work very well in Xen. For example, when I set up a small test playground, I had XCP-ng 8.3 as the primary host and another XCP-ng running on it as a nested host. When I then booted Debian 12 on the nested host, it caused the entire nested host to crash and reboot. On the other hand, Windows VMs on the nested host run quite well.

      Later, when I was preparing a test lab, I installed ESXi 8.0U3e on a Dell server and then deployed four virtualized XCP-ng 8.3 hosts on top of it. A number of Linux and Windows VMs run on them without any issues.

      M 1 Reply Last reply Reply Quote 0
      • A Offline
        abudef
        last edited by abudef

        You can enable it in the VM configuration simply by clicking a button in Xen Orchestra:

        7fc30f0d-3c2d-487c-a0f1-d235701a3762-image.jpeg

        or use this command:
        xe vm-param-set platform:nested_virt=true uuid=<VM UUID> mistake here, see below

        AlexanderKA 1 Reply Last reply Reply Quote 0
        • M Offline
          MajorP93 @abudef
          last edited by

          @abudef said:

          Nested virtualization doesn’t work very well in Xen. For example, when I set up a small test playground, I had XCP-ng 8.3 as the primary host and another XCP-ng running on it as a nested host. When I then booted Debian 12 on the nested host, it caused the entire nested host to crash and reboot. On the other hand, Windows VMs on the nested host run quite well.

          Later, when I was preparing a test lab, I installed ESXi 8.0U3e on a Dell server and then deployed four virtualized XCP-ng 8.3 hosts on top of it. A number of Linux and Windows VMs run on them without any issues.

          Did you follow the official documentation for nested virtualization?

          https://docs.xcp-ng.org/guides/xcpng-in-a-vm/#nested-xcp-ng-using-xcp-ng

          Most importantly setting via command line on pool master:

          xe vm-param-set uuid=<UUID> platform:exp-nested-hvm=true

          and

          xe vm-param-set uuid=<UUID> platform:nic_type="e1000"
          A 1 Reply Last reply Reply Quote 0
          • A Offline
            abudef @MajorP93
            last edited by

            @MajorP93 Sure I did

            M 1 Reply Last reply Reply Quote 0
            • M Offline
              MajorP93 @abudef
              last edited by

              @abudef said:

              @MajorP93 Sure I did

              Hmm you previously mentioned different command so I was not sure if you really followed documentation correctly.

              A 1 Reply Last reply Reply Quote 0
              • A Offline
                abudef @MajorP93
                last edited by

                @MajorP93 Overall it works, Windows VMs run more or less reliably, but I wasn’t able to resolve the nested host crashing when starting Debian. That’s why I put the free version of ESXi underneath the nested XCP-ng hosts, and that works reliably.

                1 Reply Last reply Reply Quote 1
                • AlexanderKA Offline
                  AlexanderK @abudef
                  last edited by

                  @abudef this causes some vms not to boot

                  A 1 Reply Last reply Reply Quote 0
                  • A Offline
                    abudef @AlexanderK
                    last edited by

                    @AlexanderK said:

                    @abudef this causes some vms not to boot

                    You mean that when you enable Nested virtualization in the VM settings under the Advanced tab, the VM then fails to boot?

                    AlexanderKA 1 Reply Last reply Reply Quote 0
                    • AlexanderKA Offline
                      AlexanderK @abudef
                      last edited by

                      @abudef exactly

                      P 1 Reply Last reply Reply Quote 0
                      • P Offline
                        p-bo @AlexanderK
                        last edited by p-bo

                        @AlexanderK Nested virtualisation still need lot of development to be functional reliably - see https://www.youtube.com/watch?v=8jKGYY1Bi_o and https://www.youtube.com/watch?v=3MxWvVTmY1s

                        1 Reply Last reply Reply Quote 0

                        Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                        Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                        With your input, this post could be even better 💗

                        Register Login
                        • First post
                          Last post