Virtualized XCP-ng reboots when trying to install Debian from an ISO from the local ISO store

abudef

Hello,
I installed XCP-ng in the virtual on XCP-ng host according to the documentation here: https://docs.xcp-ng.org/guides/xcpng-in-a-vm/#nested-xcp-ng-using-xcp-ng. Both host and guest are XCP-ng 8.3 beta 2 with all updates.

I have set up a local iso store on the virtualized XCP-ng in the /opt/iso path. I uploaded the Debian 12 installation iso (https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.5.0-amd64-netinst.iso) to this store. I created a new virtual using the Debian 12 template and used the iso. After starting, the Debian 12 installation screen appears. Unfortunately, the moment I select "Install" and press enter, the virtualized XCP-ng host immediately reboots.

Do you have any idea why the virtualized hypervisor restarts when starting a Debian installation?

best regards,
Abudef

EDIT: I tried it with a virtualized new XenServer 8 guest (on XCP-ng 8.3 host) and it behaves the same way - when starting a Debian installation from a locally stored ISO image as described above, the virtualized XenServer 8 reboots immediately.

olivierlambert

What kind of CPU are you using? AMD or Intel?

Nested can lead to this, I'm not surprised.

abudef

@olivierlambert It´s Intel i3-10110U (NUC testing mini pc). It's probably pointless to compare, but on the same HW I tried virtualizing XCP-ng on ESXi 8u2 and then Debian can be installed normally in the nested XCP-ng

olivierlambert

Because ESXi supports nested a lot more better than Xen, at least for now. Note that it wouldn't crash if you decide to use PV guests instead of HVM, but PV is less secure than HVM. I suppose that in your PV case, it might be enough though (lab/test).

abudef

Hello @lawrencesystems, I noticed in your videos that you use virtualized XCP-ng on XCP-ng for testing. What exact version are you using? Virtuals work normally in nested XCP-ng I assume?

abudef

@olivierlambert Yes, sure, PV is sufficient for testing. You mean to use PV for nested XCP-ng itself and for it´s VMs as well?

olivierlambert

Xen itself "can't be PV" but the Dom0 is, so it's for the extra VMs on top of it. Debian in PV in nested mode should work pretty well.

abudef

@olivierlambert I switched Debian VM to PV and got this error when I try to run it:

vm.start
{
  "id": "fadce86e-0f23-927a-8924-2c2ab3244779",
  "bypassMacAddressesCheck": false,
  "force": false
}
{
  "code": "INTERNAL_ERROR",
  "params": [
    "xenopsd internal error: Domain.Could_not_read_file(\"\")"
  ],
  "call": {
    "method": "VM.start",
    "params": [
      "OpaqueRef:09e98642-47ea-899f-83ee-94ea78b8bfb1",
      false,
      false
    ]
  },
  "message": "INTERNAL_ERROR(xenopsd internal error: Domain.Could_not_read_file(\"\"))",
  "name": "XapiError",
  "stack": "XapiError: INTERNAL_ERROR(xenopsd internal error: Domain.Could_not_read_file(\"\"))
    at Function.wrap (file:///opt/xo/xo-builds/xen-orchestra-202403272230/packages/xen-api/_XapiError.mjs:16:12)
    at file:///opt/xo/xo-builds/xen-orchestra-202403272230/packages/xen-api/transports/json-rpc.mjs:38:21"
}

olivierlambert

You probably have missing bootloader files. It's been a while since I did not created a PV VM. Worth creating a fresh VM in PV mode (I never converted an HVM to PV, only the opposite).

Let's see if people around it are doing it more often than me

abudef

I tried installing Ubuntu Server 22.04.4 and Windows Server 2022 Eval and both systems boot, install and work without any problems. It seems that Debian 12.5 specifically is causing the nested hypervisor to crash...

lawrencesystems

@abudef
I only did that in my video to make it easier to show how things work with XCP-ng hosts, I don't think the VM's would work inside the nested systems.

abudef

@lawrencesystems I think this is quite common when you need to test certain scenarios with multiple hypervisors (backup, migrations, etc.). You only need a couple of HVs with a few tiny running VMs. We have done this setup with nested esxi many times for testing purposes. And since e.g. Ubuntu and Windows work this way, the problem is probably specific to Debian (and maybe others?).