Imported VM Starts but Does Not Initialize the Display

[olivierlambert]

Are you sure you are booting in the same mode than the original platform? (BIOS or UEFI)

[kagbasi-ngc]

@olivierlambert Yes sir, both are configured to boot UEFI. I didn't make any changes.

I did notice, however, that in my virtualized environment (the one where I'm having the problem), the VM doesn't show the button to copy the secure boot certificates to the VM. See screenshots below.

PHYSICAL ENVIRONMENT (Where the VM boots successfully

VIRTUALIZED ENVIRONMENT (Where the VM fails to boot

[olivierlambert]

@stormi does this ring a bell?

[stormi]

Regarding the "Copy the pool's default UEFI certificates to the VM" button, I believe it is only displayed if the pool was setup for Secure Boot first. However, an imported VM comes with its own certificates, so if it was booting up correctly with Secure Boot enforced before the export, it should still do so after an import, regardless of the pool state.

Does it boot if you set Secure Boot to not enforced? I don't think it will, be let's try anyway.

[kagbasi-ngc]

@stormi I turned off Secure Boot and it's the same result. VM starts (according to the log) but doesn't proceed to POST.

[stormi]

@kagbasi-ngc So I believe this has more to do with nested virtualization. Does the XCP-ng VM have all the required virtualization capabilities?

[kagbasi-ngc]

@stormi Sorry for the delayed response. Yes, the XCP-ng VM has all the virtualization capabilities (as recommended in this guide- https://docs.xcp-ng.org/guides/xcpng-in-a-vm/). See screenshot below.

[Danp]

@kagbasi-ngc Maybe this? https://docs.xcp-ng.org/releases/release-8-3/#nested-virtualization-though-not-on-purpose

[kagbasi-ngc]

@Danp Well, that certainly puts a nail in this coffin...lol. Thanks for sharing, I'll stop testing then.

@olivierlambert should we leave this thread as unsolved until such time as nested virtualization is functional again within the codebase?

[olivierlambert]

Can you try with XCP-ng 8.2?

[stormi]

@Danp This is about hypervisors nested inside XCP-ng 8.3, but here as I understood it it was XCP-ng nested inside another hypervisor, so in theory it should not be related.

[kagbasi-ngc]

@olivierlambert So I managed to whip up a v8.2.1 instance and attempted to import the VM, but the import wouldn't start. I simply get the message in the top-right - "Starting import" - then it would go away.

I couldn't find any errors on the host logs, but I found this error within XO itself (i.e., Settings > Logs) :

My takeaway from this is that I cannot export a VM from a newer host and import it into an older one.

[olivierlambert]

Note: you cannot import a VM from a more recent XCP-ng to an older one (the opposite works though)

[kagbasi-ngc]

@olivierlambert Hmm, so why would you encourage me to try it, if you knew it wouldn't work....lol.

[olivierlambert]

I encouraged you to test if nested XCP-ng 8.2 worked better than 8.3, that's it.

[kagbasi-ngc]

@olivierlambert Okay, so I'm at fault for reading too much into your guidance, eh? Okay.

In that case then, I think we'd already established that nested virtualization works - because I have been able to add an SR, create VMs, add VDIs, etc., in the nested virtualization environment on 8.3. The only thing that was failing for me, which is the primary reason for standing it up, is the import of a VM.

So is it safe to say this is not likely an issue with nested virtualization but with something else in the codebase that's preventing the imported VM from successfully initializing the display when in UEFI mode?

[stormi]

@kagbasi-ngc I'd check the output of xl dmesg and the contents of /var/log/daemon.log and /var/log/xensource.log after trying to start the VM.

[kagbasi-ngc]

@stormi @olivierlambert

As requested, please find the outputs of the logs:

• Output of xl dmesg - https://gist.github.com/kismetgerald/403edf28d5fd358722d2bc36b52f38f1
• Output of /var/log/daemon.log - https://gist.github.com/kismetgerald/8965863047eee26b815dcfcfe4faabae
• Output of /var/log/xensource.log - https://gist.github.com/kismetgerald/4ed62e999d9920d697f06c8e42de9873

For time reference, I started the VM at approximately Dec 12 05:08 - don't recall the precise second. If I can provide other logs, please let me know.

[stormi]

@kagbasi-ngc I was inviting you to have a look first, but I'll try to give it a look. Still, try to find something relevant in them, it's a good troubleshooting exercise.

[stormi]

Relevant logs, from daemon.log:

Dec 12 05:08:18 VMH01 qemu-dm-1[8691]: SyncPcrAllocationsAndPcrMask!
Dec 12 05:08:18 VMH01 qemu-dm-1[8691]: Set PcdTpm2Hash Mask to 0x0000000F
Dec 12 05:08:19 VMH01 qemu-dm-1[8691]: AllocatePages failed: No 0x8400 Pages is available.
Dec 12 05:08:19 VMH01 qemu-dm-1[8691]: There is only left 0x3AA8 pages memory resource to be allocated.
Dec 12 05:08:19 VMH01 qemu-dm-1[8691]: ERROR: Out of aligned pages
Dec 12 05:08:19 VMH01 qemu-dm-1[8691]: ASSERT /builddir/build/BUILD/edk2-20220801/MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c(814): BigPageAddress != 0

ERROR: Out of aligned pages does not look good to me.