XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    "Hardware-assisted virtualization is not enabled on this host" even though platform:exp-nested-hvm=true is set

    Scheduled Pinned Locked Moved XCP-ng
    58 Posts 7 Posters 12.6k Views 8 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • abudefA Offline
      abudef @Danp
      last edited by

      @Danp Yes, I did.

      DanpD 1 Reply Last reply Reply Quote 0
      • DanpD Offline
        Danp Pro Support Team @abudef
        last edited by

        @abudef Is the vm.start above from attempting to start the nested XCP-ng host?

        abudefA 1 Reply Last reply Reply Quote 0
        • abudefA Offline
          abudef @Danp
          last edited by

          @Danp said in "Hardware-assisted virtualization is not enabled on this host" even though platform:exp-nested-hvm=true is set:

          @abudef Is the vm.start above from attempting to start the nested XCP-ng host?

          No, it's from trying to run a Debian VM on a virtualized nested XCP-ng host.

          1 Reply Last reply Reply Quote 0
          • olivierlambertO Offline
            olivierlambert Vates 🪐 Co-Founder CEO
            last edited by

            Nested isn't an exact science sadly. But:

            1. Double check you have enabled it at your XCP-ng VM level
            2. Check if your BIOS or hardware is supporting nested
            abudefA 1 Reply Last reply Reply Quote 0
            • abudefA Offline
              abudef @olivierlambert
              last edited by

              @olivierlambert I checked it carefully and I don't see a problem in the configuration and settings:

              1. This is the server running the base XCP-ng host:
                635ef8d5-e41f-4885-b0cf-0ae7dad1013d-obrazek.png

              2. There is a virtual machine on this host, based on a CentOS 7 template with the necessary settings:
                7b524bec-b8f7-4a7d-b93f-7fda7eae4c45-obrazek.png

              3. If I then try to install the nested XCP-ng on it, it already reports that virtualization is not supported during installation:
                720f4bd9-580a-4714-a8e2-25a8e54a986e-obrazek.png

              1 Reply Last reply Reply Quote 0
              • olivierlambertO Offline
                olivierlambert Vates 🪐 Co-Founder CEO
                last edited by

                That looks good on the XO side. Let's check on the XAPI side, please do a xe vm-param-list uuid=<XCP-ng nested VM UUID> and paste it here.

                I would also double check in BIOS that you have IOMMU, VT-d and other similar options enabled.

                abudefA 1 Reply Last reply Reply Quote 0
                • abudefA Offline
                  abudef @olivierlambert
                  last edited by

                  @olivierlambert BIOS looks good and with ESXi it worked before on this machine.

                  uuid ( RO)                                  : f182cd5a-846a-0bd4-feab-ac8c40d1b4b5
                                              name-label ( RW): XCP-ng
                                        name-description ( RW):
                                            user-version ( RW): 1
                                           is-a-template ( RW): false
                                     is-default-template ( RW): false
                                           is-a-snapshot ( RO): false
                                             snapshot-of ( RO): <not in database>
                                               snapshots ( RO):
                                           snapshot-time ( RO): 19700101T00:00:00Z
                                           snapshot-info ( RO):
                                                  parent ( RO): <not in database>
                                                children ( RO):
                                       is-control-domain ( RO): false
                                             power-state ( RO): halted
                                           memory-actual ( RO): 0
                                           memory-target ( RO): 0
                                         memory-overhead ( RO): 547356672
                                       memory-static-max ( RW): 68719476736
                                      memory-dynamic-max ( RW): 68719476736
                                      memory-dynamic-min ( RW): 68719476736
                                       memory-static-min ( RW): 2147483648
                                        suspend-VDI-uuid ( RW): <not in database>
                                         suspend-SR-uuid ( RW): <not in database>
                                            VCPUs-params (MRW):
                                               VCPUs-max ( RW): 8
                                        VCPUs-at-startup ( RW): 8
                                  actions-after-shutdown ( RW): Destroy
                                actions-after-softreboot ( RW): Soft reboot
                                    actions-after-reboot ( RW): Restart
                                     actions-after-crash ( RW): Restart
                                           console-uuids (SRO):
                                                     hvm ( RO): false
                                                platform (MRW): timeoffset: 0; device-model: qemu-upstream-uefi; secureboot: false; exp-nested-hvm: true; nic_type: e1000; vga: std; videoram: 8; viridian: false; device_id: 0001; nx: true; acpi: 1; apic: true; pae: true; hpet: true
                                      allowed-operations (SRO): create_vtpm; changing_NVRAM; changing_dynamic_range; changing_shadow_memory; changing_static_range; make_into_template; migrate_send; destroy; export; start_on; start; clone; copy; snapshot
                                      current-operations (SRO):
                                      blocked-operations (MRW):
                                     allowed-VBD-devices (SRO): 1; 2; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49; 50; 51; 52; 53; 54; 55; 56; 57; 58; 59; 60; 61; 62; 63; 64; 65; 66; 67; 68; 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89; 90; 91; 92; 93; 94; 95; 96; 97; 98; 99; 100; 101; 102; 103; 104; 105; 106; 107; 108; 109; 110; 111; 112; 113; 114; 115; 116; 117; 118; 119; 120; 121; 122; 123; 124; 125; 126; 127; 128; 129; 130; 131; 132; 133; 134; 135; 136; 137; 138; 139; 140; 141; 142; 143; 144; 145; 146; 147; 148; 149; 150; 151; 152; 153; 154; 155; 156; 157; 158; 159; 160; 161; 162; 163; 164; 165; 166; 167; 168; 169; 170; 171; 172; 173; 174; 175; 176; 177; 178; 179; 180; 181; 182; 183; 184; 185; 186; 187; 188; 189; 190; 191; 192; 193; 194; 195; 196; 197; 198; 199; 200; 201; 202; 203; 204; 205; 206; 207; 208; 209; 210; 211; 212; 213; 214; 215; 216; 217; 218; 219; 220; 221; 222; 223; 224; 225; 226; 227; 228; 229; 230; 231; 232; 233; 234; 235; 236; 237; 238; 239; 240; 241; 242; 243; 244; 245; 246; 247; 248; 249; 250; 251; 252; 253; 254
                                     allowed-VIF-devices (SRO): 1; 2; 3; 4; 5; 6
                                          possible-hosts ( RO): 43db4c0b-9449-41df-b301-5de08cede1e7
                                             domain-type ( RW): hvm
                                     current-domain-type ( RO): unspecified
                                         HVM-boot-policy ( RW): BIOS order
                                         HVM-boot-params (MRW): firmware: uefi; order: cdn
                                   HVM-shadow-multiplier ( RW): 1.000
                                               PV-kernel ( RW):
                                              PV-ramdisk ( RW):
                                                 PV-args ( RW):
                                          PV-legacy-args ( RW):
                                           PV-bootloader ( RW):
                                      PV-bootloader-args ( RW):
                                     last-boot-CPU-flags ( RO): vendor: GenuineIntel; features:
                                        last-boot-record ( RO): ''
                                             resident-on ( RO): <not in database>
                                                affinity ( RW): <not in database>
                                            other-config (MRW): xo:f182cd5a: {"creation":{"date":"2024-05-01T07:09:40.657Z","template":"11fd3dc9-96cc-49af-b091-a2ca7e94c589","user":"58b144ce-054d-4438-8fc7-154a0160807a"}}; base_template_name: CentOS 7; import_task: OpaqueRef:ed4e8a12-3781-fe9c-3bbb-77fd5b719ec1; mac_seed: 3e9a111c-a830-9c04-d1ca-ab4018f6d889; linux_template: true; install-methods: cdrom,nfs,http,ftp
                                                  dom-id ( RO): -1
                                         recommendations ( RO): <restrictions><restriction field="memory-static-max" max="1649267441664"/><restriction field="vcpus-max" max="64"/><restriction field="has-vendor-device" value="false"/><restriction field="allow-gpu-passthrough" value="1"/><restriction field="allow-vgpu" value="1"/><restriction field="allow-network-sriov" value="1"/><restriction field="supports-bios" value="yes"/><restriction field="supports-uefi" value="no"/><restriction field="supports-secure-boot" value="no"/><restriction max="255" property="number-of-vbds"/><restriction max="7" property="number-of-vifs"/></restrictions>
                                           xenstore-data (MRW): vm-data/mmio-hole-size: 268435456; vm-data:
                              ha-always-run ( RW) [DEPRECATED]: false
                                     ha-restart-priority ( RW):
                                                   blobs ( RO):
                                              start-time ( RO): 19700101T00:00:00Z
                                            install-time ( RO): 20240501T07:09:40Z
                                            VCPUs-number ( RO): 0
                                       VCPUs-utilisation (MRO):
                                              os-version (MRO):
                                      PV-drivers-version (MRO):
                      PV-drivers-up-to-date ( RO) [DEPRECATED]: false
                                                  memory (MRO):
                                                   disks (MRO):
                                                    VBDs (SRO): 3fe5211d-07c4-9a6c-5f28-cd3a8caffb8e; 760be722-f754-e2f9-f1d6-a6e800128356
                                                networks (MRO):
                                     PV-drivers-detected ( RO): false
                                                   other (MRO): platform-feature-xs_reset_watches: 1; platform-feature-multiprocessor-suspend: 1; has-vendor-device: 0
                                                    live ( RO): true
                              guest-metrics-last-updated ( RO): 20240501T07:23:03Z
                                     can-use-hotplug-vbd ( RO): unspecified
                                     can-use-hotplug-vif ( RO): unspecified
                                cooperative ( RO) [DEPRECATED]: true
                                                    tags (SRW):
                                               appliance ( RW): <not in database>
                                       snapshot-schedule ( RW): <not in database>
                                        is-vmss-snapshot ( RO): false
                                             start-delay ( RW): 0
                                          shutdown-delay ( RW): 0
                                                   order ( RW): 0
                                                 version ( RO): 0
                                           generation-id ( RO):
                               hardware-platform-version ( RO): 0
                                       has-vendor-device ( RW): false
                                         requires-reboot ( RO): false
                                         reference-label ( RO): centos-7
                                            bios-strings (MRO): bios-vendor: Xen; bios-version: ; system-manufacturer: Xen; system-product-name: HVM domU; system-version: ; system-serial-number: ; baseboard-manufacturer: ; baseboard-product-name: ; baseboard-version: ; baseboard-serial-number: ; baseboard-asset-tag: ; baseboard-location-in-chassis: ; enclosure-asset-tag: ; hp-rombios: ; oem-1: Xen; oem-2: MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d
                                       pending-guidances ( RO):
                                                   vtpms ( RO):
                           pending-guidances-recommended ( RO):
                                  pending-guidances-full ( RO):
                  
                  
                  1 Reply Last reply Reply Quote 0
                  • olivierlambertO Offline
                    olivierlambert Vates 🪐 Co-Founder CEO
                    last edited by

                    @abudef said in "Hardware-assisted virtualization is not enabled on this host" even though platform:exp-nested-hvm=true is set:

                    exp-nested-hvm: true;

                    So you have the good parameter here. Maybe try in BIOS mode to see if it's better?

                    abudefA 1 Reply Last reply Reply Quote 0
                    • abudefA Offline
                      abudef @olivierlambert
                      last edited by abudef

                      It behaves exactly the same regardless of whether BIOS or UEFI is used.

                      1 Reply Last reply Reply Quote 0
                      • abudefA Offline
                        abudef
                        last edited by

                        But I have a suspicion that it hasn't worked since the last or second to last batch of updates for XCP-ng, that it worked before.

                        abudefA 1 Reply Last reply Reply Quote 1
                        • abudefA Offline
                          abudef @abudef
                          last edited by abudef

                          But I have a suspicion that it hasn't worked since the last or second to last batch of updates for XCP-ng, that it worked before.

                          I can confirm this; on another lab I updated XCP-ng with the last fifty or so updates from the end of April, and nested XCP-ng also appears to have virtualization support not enabled...

                          1 Reply Last reply Reply Quote 0
                          • olivierlambertO Offline
                            olivierlambert Vates 🪐 Co-Founder CEO
                            last edited by

                            You mean 8.2 or 8.3?

                            abudefA 1 Reply Last reply Reply Quote 0
                            • abudefA Offline
                              abudef @olivierlambert
                              last edited by abudef

                              I use version 8.3.

                              1 Reply Last reply Reply Quote 0
                              • olivierlambertO Offline
                                olivierlambert Vates 🪐 Co-Founder CEO
                                last edited by

                                That might be interesting to check if it works with the same context on 8.2

                                1 Reply Last reply Reply Quote 0
                                • abudefA Offline
                                  abudef
                                  last edited by

                                  Nested virtualization seems to work correctly in version 8.2.1 with the latest updates (native host).

                                  abudefA 1 Reply Last reply Reply Quote 0
                                  • abudefA Offline
                                    abudef @abudef
                                    last edited by abudef

                                    Nested virtualization seems to work correctly in version 8.2.1 with the latest updates (native host).

                                    On the contrary; the problem is related to virtualized XCP-ng version 8.3 with the latest updates, not to the native host.

                                    abudefA 1 Reply Last reply Reply Quote 0
                                    • abudefA Offline
                                      abudef @abudef
                                      last edited by

                                      Nested virtualization seems to work correctly in version 8.2.1 with the latest updates (native host).

                                      On the contrary; the problem is related to virtualized XCP-ng version 8.3 with the latest updates, not to the native host.

                                      The combination (8.3latest(8.x)) does not work either. Nested virtualization simply doesn't work if 8.3 is either a native host, a virtualized host, or both.

                                      1 Reply Last reply Reply Quote 0
                                      • olivierlambertO Offline
                                        olivierlambert Vates 🪐 Co-Founder CEO
                                        last edited by

                                        And 8.3 with Xen 4.13 instead of 4.17?

                                        1 Reply Last reply Reply Quote 0
                                        • abudefA Offline
                                          abudef
                                          last edited by

                                          Well, it seems that the problem is caused by Xen 4.17...

                                          1 Reply Last reply Reply Quote 0
                                          • abudefA Offline
                                            abudef
                                            last edited by

                                            I temporarily moved the entire XCP-ng lab to ESXi. I need a few different hosts for scenario testing, migration and backup testing, XO proxies testing etc. and of course I can't dedicate a hw server for each of them in the lab. Please let me know when the nested virtualization support will be operational again on version 8.3.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post