I found the same message in the Backup log as well:
Xen Orchestra, commit 9b9c7
Several VMs are being backed up, this message only appears for one.
I found the same message in the Backup log as well:
Xen Orchestra, commit 9b9c7
Several VMs are being backed up, this message only appears for one.
@olivierlambert @stormi Please do you have any idea when this problem might be resolved? The question is how to deal with the test lab, whether to wait, because a secondary problem is that the nested VMs cannot be migrated elsewhere from the affected virtualized XCP-ng hosts.
vm.migrate
{
"vm": "654cc5c6-7e50-fc28-ecc4-fe46929905b2",
"mapVifsNetworks": {
"2db4235a-345f-f286-4172-77dab4e198fe
": "8e969c1a-cafa-7ac0-504d-cf5cd19ef1e4
"
},
"migrationNetwork": "8e969c1a-cafa-7ac0-504d-cf5cd19ef1e4
",
"sr": "a25ba333-a1a5-f22f-c337-0ec662e835ed",
"targetHost": "ca60fce7-924a-45f9-a1c6-ee860952e6aa"
}
{
"code": "NO_HOSTS_AVAILABLE",
"params": [],
"task": {
"uuid": "57fe6efb-569e-3fa3-a345-d43377260884
",
"name_label": "Async.VM.migrate_send",
"name_description": "",
"allowed_operations": [],
"current_operations": {},
"created": "20240510T10:45:50Z",
"finished": "20240510T10:45:51Z",
"status": "failure",
"resident_on": "OpaqueRef:99de1bb8-8e7f-e79a-d7f9-5d84c2c09a73",
"progress": 1,
"type": "<none/>",
"result": "",
"error_info": [
"NO_HOSTS_AVAILABLE"
],
"other_config": {},
"subtask_of": "OpaqueRef:NULL",
"subtasks": [],
"backtrace": "(((process xapi)(filename ocaml/xapi/xapi_vm_placement.ml)(line 106))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 1453))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/helpers.ml)(line 1506))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 1445))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 2537))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 2559))((process xapi)(filename ocaml/xapi/rbac.ml)(line 189))((process xapi)(filename ocaml/xapi/rbac.ml)(line 198))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 75)))"
},
"message": "NO_HOSTS_AVAILABLE()",
"name": "XapiError",
"stack": "XapiError: NO_HOSTS_AVAILABLE()
at Function.wrap (file:///opt/xo/xo-builds/xen-orchestra-202405091612/packages/xen-api/_XapiError.mjs:16:12)
at default (file:///opt/xo/xo-builds/xen-orchestra-202405091612/packages/xen-api/_getTaskResult.mjs:11:29)
at Xapi._addRecordToCache (file:///opt/xo/xo-builds/xen-orchestra-202405091612/packages/xen-api/index.mjs:1029:24)
at file:///opt/xo/xo-builds/xen-orchestra-202405091612/packages/xen-api/index.mjs:1063:14
at Array.forEach (<anonymous>)
at Xapi._processEvents (file:///opt/xo/xo-builds/xen-orchestra-202405091612/packages/xen-api/index.mjs:1053:12)
at Xapi._watchEvents (file:///opt/xo/xo-builds/xen-orchestra-202405091612/packages/xen-api/index.mjs:1226:14)"
}
(targetHost above is a common native XCP-ng host)
As expected, the same problem is with XenServer 8/4.17
I was wondering if it is possible to install guest tools in a nested XCP-ng running in XCP-ng? In order to monitor and control such a virtual machine.
Disable nested from the VM/Advanced tab
xe vm-param-set platform:nested_virt=true uuid=<VM UUID>
Start the VM
...unfortunately it didn't work, I assume because...
It seems we also need to modify some other bits
@stormi That's not quite what I meant. The point is that if the host is 8.3/4.17, then virtualization is not working on its guest and therefore no other nested guest can be started on this guest. And actually, yes, if 8.3/4.17 appears somewhere in the cascade of nested hosts, then nested virtualization will no longer work on its guest.
@stormi If the host is on version 8.3 with Xen 4.17, nested virtualization does not work on any guest. If version 8.3 with Xen 4.17 is on a guest, but the host is running version 8.2.1 or an out-of-date 8.3 (Xen 4.13), it doesn't matter and nested guest virtualization works.
@olivierlambert Thank you very much for the impeccable support and fast communication
I tested again today and the problem is always on the host side, regardless of the type of guest. This corrects some of my hasty information above.
I temporarily moved the entire XCP-ng lab to ESXi. I need a few different hosts for scenario testing, migration and backup testing, XO proxies testing etc. and of course I can't dedicate a hw server for each of them in the lab. Please let me know when the nested virtualization support will be operational again on version 8.3.
Well, it seems that the problem is caused by Xen 4.17...
Nested virtualization seems to work correctly in version 8.2.1 with the latest updates (
native host).On the contrary; the problem is related to virtualized XCP-ng version 8.3 with the latest updates, not to the native host.
The combination (8.3latest(8.x)) does not work either. Nested virtualization simply doesn't work if 8.3 is either a native host, a virtualized host, or both.
Nested virtualization seems to work correctly in version 8.2.1 with the latest updates (
native host).
On the contrary; the problem is related to virtualized XCP-ng version 8.3 with the latest updates, not to the native host.
Nested virtualization seems to work correctly in version 8.2.1 with the latest updates (native host).
But I have a suspicion that it hasn't worked since the last or second to last batch of updates for XCP-ng, that it worked before.
I can confirm this; on another lab I updated XCP-ng with the last fifty or so updates from the end of April, and nested XCP-ng also appears to have virtualization support not enabled...
But I have a suspicion that it hasn't worked since the last or second to last batch of updates for XCP-ng, that it worked before.
It behaves exactly the same regardless of whether BIOS or UEFI is used.
@olivierlambert BIOS looks good and with ESXi it worked before on this machine.
uuid ( RO) : f182cd5a-846a-0bd4-feab-ac8c40d1b4b5
name-label ( RW): XCP-ng
name-description ( RW):
user-version ( RW): 1
is-a-template ( RW): false
is-default-template ( RW): false
is-a-snapshot ( RO): false
snapshot-of ( RO): <not in database>
snapshots ( RO):
snapshot-time ( RO): 19700101T00:00:00Z
snapshot-info ( RO):
parent ( RO): <not in database>
children ( RO):
is-control-domain ( RO): false
power-state ( RO): halted
memory-actual ( RO): 0
memory-target ( RO): 0
memory-overhead ( RO): 547356672
memory-static-max ( RW): 68719476736
memory-dynamic-max ( RW): 68719476736
memory-dynamic-min ( RW): 68719476736
memory-static-min ( RW): 2147483648
suspend-VDI-uuid ( RW): <not in database>
suspend-SR-uuid ( RW): <not in database>
VCPUs-params (MRW):
VCPUs-max ( RW): 8
VCPUs-at-startup ( RW): 8
actions-after-shutdown ( RW): Destroy
actions-after-softreboot ( RW): Soft reboot
actions-after-reboot ( RW): Restart
actions-after-crash ( RW): Restart
console-uuids (SRO):
hvm ( RO): false
platform (MRW): timeoffset: 0; device-model: qemu-upstream-uefi; secureboot: false; exp-nested-hvm: true; nic_type: e1000; vga: std; videoram: 8; viridian: false; device_id: 0001; nx: true; acpi: 1; apic: true; pae: true; hpet: true
allowed-operations (SRO): create_vtpm; changing_NVRAM; changing_dynamic_range; changing_shadow_memory; changing_static_range; make_into_template; migrate_send; destroy; export; start_on; start; clone; copy; snapshot
current-operations (SRO):
blocked-operations (MRW):
allowed-VBD-devices (SRO): 1; 2; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26; 27; 28; 29; 30; 31; 32; 33; 34; 35; 36; 37; 38; 39; 40; 41; 42; 43; 44; 45; 46; 47; 48; 49; 50; 51; 52; 53; 54; 55; 56; 57; 58; 59; 60; 61; 62; 63; 64; 65; 66; 67; 68; 69; 70; 71; 72; 73; 74; 75; 76; 77; 78; 79; 80; 81; 82; 83; 84; 85; 86; 87; 88; 89; 90; 91; 92; 93; 94; 95; 96; 97; 98; 99; 100; 101; 102; 103; 104; 105; 106; 107; 108; 109; 110; 111; 112; 113; 114; 115; 116; 117; 118; 119; 120; 121; 122; 123; 124; 125; 126; 127; 128; 129; 130; 131; 132; 133; 134; 135; 136; 137; 138; 139; 140; 141; 142; 143; 144; 145; 146; 147; 148; 149; 150; 151; 152; 153; 154; 155; 156; 157; 158; 159; 160; 161; 162; 163; 164; 165; 166; 167; 168; 169; 170; 171; 172; 173; 174; 175; 176; 177; 178; 179; 180; 181; 182; 183; 184; 185; 186; 187; 188; 189; 190; 191; 192; 193; 194; 195; 196; 197; 198; 199; 200; 201; 202; 203; 204; 205; 206; 207; 208; 209; 210; 211; 212; 213; 214; 215; 216; 217; 218; 219; 220; 221; 222; 223; 224; 225; 226; 227; 228; 229; 230; 231; 232; 233; 234; 235; 236; 237; 238; 239; 240; 241; 242; 243; 244; 245; 246; 247; 248; 249; 250; 251; 252; 253; 254
allowed-VIF-devices (SRO): 1; 2; 3; 4; 5; 6
possible-hosts ( RO): 43db4c0b-9449-41df-b301-5de08cede1e7
domain-type ( RW): hvm
current-domain-type ( RO): unspecified
HVM-boot-policy ( RW): BIOS order
HVM-boot-params (MRW): firmware: uefi; order: cdn
HVM-shadow-multiplier ( RW): 1.000
PV-kernel ( RW):
PV-ramdisk ( RW):
PV-args ( RW):
PV-legacy-args ( RW):
PV-bootloader ( RW):
PV-bootloader-args ( RW):
last-boot-CPU-flags ( RO): vendor: GenuineIntel; features:
last-boot-record ( RO): ''
resident-on ( RO): <not in database>
affinity ( RW): <not in database>
other-config (MRW): xo:f182cd5a: {"creation":{"date":"2024-05-01T07:09:40.657Z","template":"11fd3dc9-96cc-49af-b091-a2ca7e94c589","user":"58b144ce-054d-4438-8fc7-154a0160807a"}}; base_template_name: CentOS 7; import_task: OpaqueRef:ed4e8a12-3781-fe9c-3bbb-77fd5b719ec1; mac_seed: 3e9a111c-a830-9c04-d1ca-ab4018f6d889; linux_template: true; install-methods: cdrom,nfs,http,ftp
dom-id ( RO): -1
recommendations ( RO): <restrictions><restriction field="memory-static-max" max="1649267441664"/><restriction field="vcpus-max" max="64"/><restriction field="has-vendor-device" value="false"/><restriction field="allow-gpu-passthrough" value="1"/><restriction field="allow-vgpu" value="1"/><restriction field="allow-network-sriov" value="1"/><restriction field="supports-bios" value="yes"/><restriction field="supports-uefi" value="no"/><restriction field="supports-secure-boot" value="no"/><restriction max="255" property="number-of-vbds"/><restriction max="7" property="number-of-vifs"/></restrictions>
xenstore-data (MRW): vm-data/mmio-hole-size: 268435456; vm-data:
ha-always-run ( RW) [DEPRECATED]: false
ha-restart-priority ( RW):
blobs ( RO):
start-time ( RO): 19700101T00:00:00Z
install-time ( RO): 20240501T07:09:40Z
VCPUs-number ( RO): 0
VCPUs-utilisation (MRO):
os-version (MRO):
PV-drivers-version (MRO):
PV-drivers-up-to-date ( RO) [DEPRECATED]: false
memory (MRO):
disks (MRO):
VBDs (SRO): 3fe5211d-07c4-9a6c-5f28-cd3a8caffb8e; 760be722-f754-e2f9-f1d6-a6e800128356
networks (MRO):
PV-drivers-detected ( RO): false
other (MRO): platform-feature-xs_reset_watches: 1; platform-feature-multiprocessor-suspend: 1; has-vendor-device: 0
live ( RO): true
guest-metrics-last-updated ( RO): 20240501T07:23:03Z
can-use-hotplug-vbd ( RO): unspecified
can-use-hotplug-vif ( RO): unspecified
cooperative ( RO) [DEPRECATED]: true
tags (SRW):
appliance ( RW): <not in database>
snapshot-schedule ( RW): <not in database>
is-vmss-snapshot ( RO): false
start-delay ( RW): 0
shutdown-delay ( RW): 0
order ( RW): 0
version ( RO): 0
generation-id ( RO):
hardware-platform-version ( RO): 0
has-vendor-device ( RW): false
requires-reboot ( RO): false
reference-label ( RO): centos-7
bios-strings (MRO): bios-vendor: Xen; bios-version: ; system-manufacturer: Xen; system-product-name: HVM domU; system-version: ; system-serial-number: ; baseboard-manufacturer: ; baseboard-product-name: ; baseboard-version: ; baseboard-serial-number: ; baseboard-asset-tag: ; baseboard-location-in-chassis: ; enclosure-asset-tag: ; hp-rombios: ; oem-1: Xen; oem-2: MS_VM_CERT/SHA1/bdbeb6e0a816d43fa6d3fe8aaef04c2bad9d3e3d
pending-guidances ( RO):
vtpms ( RO):
pending-guidances-recommended ( RO):
pending-guidances-full ( RO):
@olivierlambert I checked it carefully and I don't see a problem in the configuration and settings:
This is the server running the base XCP-ng host:
There is a virtual machine on this host, based on a CentOS 7 template with the necessary settings:
If I then try to install the nested XCP-ng on it, it already reports that virtualization is not supported during installation:
@Danp said in "Hardware-assisted virtualization is not enabled on this host" even though platform:exp-nested-hvm=true is set:
@abudef Is the
vm.start
above from attempting to start the nested XCP-ng host?
No, it's from trying to run a Debian VM on a virtualized nested XCP-ng host.