When we released XCP-ng 8.3, migration from XenServer 8 did work. However, XS8 has kept on evolving, rolling-release style, so it's possible that the current installation ISOs for XCP-ng 8.3 are not recent enough to migrate a current day XenServer 8.

However, the problems I would expect are not the ones you describe. I would expect the upgrade process to go smoothtly, and then the XAPI service fail to start because our version of XAPI sees a database that is too recent.

If no choices are displayed to upgrade an existing XenServer 8 with the 8.3 ISO, the reason will be provided in the logs in /tmp/install-log (ALT+RIGHT then view /tmp/install-log, IIRC). It can be several things: a first boot process not complete on XS side, booting in BIOS mode when the system is currently UEFI, ... But there's also the possibility that the platform number we use to detect an upgradeable product has changed.

@Mefosheez Marking a new post as a question will not be necessary. @anthonyper is the one working on what we think is the issue (a regression on the EDK2 package). I haven't heard of a fix yet, but it's in progress.

XenServer 8 is actually XenServer 8.3 (or 8.4 but let's not complicate things, for our matter here 8.3 is good enough).

So you can migrate it to XCP-ng 8.3 but not to XCP-ng 8.2.1.

I think this part of the doc describes your issue: https://docs.xcp-ng.org/releases/release-8-3/#a-uefi-vm-started-once-on-xcp-ng-83-cant-start-if-moved-back-to-xcp-ng-821

I'm sure we have forum threads about I210 already, but the forum's search engine is as useless as usual, trying to be smarter and not respecting search words when they're not in its dictionary

The Linux driver for I210 is igb and I think it usually works.

Which version of XCP-ng are you trying?

We have an intel-igb-alt driver RPM that you can install (see https://docs.xcp-ng.org/installation/hardware/#alternate-drivers). On 8.2, it provides a newer version of the driver. On XCP-ng 8.3, on the contrary, you get an older version, but that can still be an interesting test. See https://github.com/xcp-ng/xcp/wiki/Drivers for the available driver versions.

Pinging @bleader for the network part, with help from @gduperrey to navigate among the existing drivers. @Andrew might have pieces of insight, too.

@Mefosheez I forwarded internally. Feel free to ask again after a week if no answer comes.

What does mdadm --detail /dev/md127 output on either hosts?

@kagbasi-ngc Does your VM have a vTPM? If so, could you try without?

@kagbasi-ngc @Mefosheez We're trying to find developer time to diagnose the cause, but it's not the best time of the year, so I can't promise anything on delays.

@kagbasi-ngc Looks like we missed something, as the log indicates it loaded OVMF-release.fd:

Dec 13 13:54:59 VMH01 xenguest-2-build[8345]: Loaded OVMF from /usr/share/edk2/OVMF-release.fd

We'll come back to you with a better test procedure.

For now I just have a quick and dirty one, consisting in overwriting the release file with the debug one:

cp /usr/share/edk2/OVMF-debug.fd /usr/share/edk2/OVMF-release.fd

You can revert this change with yum reinstall edk2.

I'm not spooked, I just think it's the relevant error message we need to understand :).

Since this is a test pool, could you make it use the debug version of OVMF?

ln -sf OVMF-debug.fd /usr/share/edk2/OVMF.fd 

Then attempt to start the VM again, and get /var/log/daemon.log and /var/log/xensource.log once again.

Relevant logs, from daemon.log:

Dec 12 05:08:18 VMH01 qemu-dm-1[8691]: SyncPcrAllocationsAndPcrMask!
Dec 12 05:08:18 VMH01 qemu-dm-1[8691]: Set PcdTpm2Hash Mask to 0x0000000F
Dec 12 05:08:19 VMH01 qemu-dm-1[8691]: AllocatePages failed: No 0x8400 Pages is available.
Dec 12 05:08:19 VMH01 qemu-dm-1[8691]: There is only left 0x3AA8 pages memory resource to be allocated.
Dec 12 05:08:19 VMH01 qemu-dm-1[8691]: ERROR: Out of aligned pages
Dec 12 05:08:19 VMH01 qemu-dm-1[8691]: ASSERT /builddir/build/BUILD/edk2-20220801/MdeModulePkg/Core/DxeIplPeim/X64/VirtualMemory.c(814): BigPageAddress != 0

ERROR: Out of aligned pages does not look good to me.

@kagbasi-ngc I was inviting you to have a look first, but I'll try to give it a look. Still, try to find something relevant in them, it's a good troubleshooting exercise.

I assumed you wanted live migration, but if you can do with a few minutes of downtime, you can also do a warm migration: https://xen-orchestra.com/blog/warm-migration-with-xen-orchestra/

@OlegF I can't promise anything. It's designed to work, but sometimes, with such a big jump, VMs can crash. Especially if there are unfixed bugs in the sender host that manifest themselves only when migrating to some more recent versions (this happened in the past with cross pool migrations from XenServer <7.1 towards more recent pools).

@kagbasi-ngc I'd check the output of xl dmesg and the contents of /var/log/daemon.log and /var/log/xensource.log after trying to start the VM.

@Danp This is about hypervisors nested inside XCP-ng 8.3, but here as I understood it it was XCP-ng nested inside another hypervisor, so in theory it should not be related.

@kagbasi-ngc So I believe this has more to do with nested virtualization. Does the XCP-ng VM have all the required virtualization capabilities?

Regarding the "Copy the pool's default UEFI certificates to the VM" button, I believe it is only displayed if the pool was setup for Secure Boot first. However, an imported VM comes with its own certificates, so if it was booting up correctly with Secure Boot enforced before the export, it should still do so after an import, regardless of the pool state.

Does it boot if you set Secure Boot to not enforced? I don't think it will, be let's try anyway.

We are implementing tests in our CI to detect this issue and protect against regressions, then we'll investigate the cause.