@Chico008 said in XOA - Vm linux install keep failing since updateing XCP to 8.3:

tried using uefi (non secure) or bios, each time install failes.

ISOs comme from my NAS, and VM disk are hosted on a ISCSI Lun

on 8.2 was exactly the same conf, sources, all worked fine, but now i can't :/, and i'm unable to understand the report, i only it failed during kernel install.

my Isos are the same i used for 8.2, on the same NFS Share.

tried installing a win 2K19 server, seems ok, but Ubuntu server can't (even tried a ubuntu server 22, same)

Reply

Could you post the ISO version you were using for the installation and hardware details?

@acebmxer Could you fetch and send the error report somewhere?

@lastcmaster Hi, it's a known issue that guest agent versions are not reported after migration or suspend. Other functionalities (poweroff/reboot, suspend, network change etc.) should continue to work normally.

@flakpyro said in VM start stuck on "Guest has not initialized the display (yet).":

@dinhngtu said in VM start stuck on "Guest has not initialized the display (yet).":

You must run secureboot-certs clear if you're updating from 1.2.0-2.4 or 1.2.0-3.1 and have previously run secureboot-certs install with the above versions installed.

Should we run this before installing the update or after 1.2.0-3.2 has been installed?

You should run that preferably after updating all hosts.

@lukasz_s said in VM start stuck on "Guest has not initialized the display (yet).":

@dinhngtu thanks for advice

i've upgraded varstored and varstored-tools:

rpm -qa | grep varstored
varstored-1.2.0-3.2
varstored-tools-1.2.0-3.2

than i've cleared varstore with secureboot-certs clear
should taht folder contain more files ?

ls /usr/share/varstored/
KEK.uth
PK.auth
db.auth

what about dbx file ?

That file is not shipped with varstored nor needed for now. We're validating the final 1.2.0-3.2 and preparing our guidance for the official update.

@acebmxer All looks good from the output. There might be some VMs that need fixes for Windows dbx updates to work properly, I'll try to add that in the official version.

@acebmxer Yes, that'll fix the situation.

Note that we're discussing a different approach for the official update, which will involve using a manual repair tool.

@flakpyro said in XCP-ng 8.3 updates announcements and testing:

@dinhngtu The output of that command seems to result in the output of "Scanningn every UUID in my pool", most of which do not use SB. Im guessing that normal and i do not have any affected VMs any longer?

Yes, it'll produce an error message in unserialize_variables when there's a problem.

@acebmxer Scanning on 1 host will scan the whole pool at once.

In the output you posted, there's only c81f1f1b-9f95-38ec-f96e-5ce8fea073e6 which needs fixing.

@acebmxer You can confirm if there are other affected VMs using this command:

xe vm-list --minimal | xargs -d, -Ix sh -c "echo Scanning x 1>&2; varstore-ls x" > /dev/null

If there are still affected VMs, you must fix them on varstored-1.2.0-3.2 by secureboot-certs clear + propagating before downgrading to 2.3 (if you wish).

Note that downgrading to 2.3 with pool certs cleared means you'll need to set up guest Secure Boot again. So I recommend staying on 3.2 if it's possible for you.

@lukasz_s Thanks. As this is a test update, please take note of any issues especially with new Windows installations.

For reference, here are the prerelease notes:

• Both varstored and varstored-tools must be updated together.
• After the update, to ensure that the new varstored version is running, VMs must be migrated to an updated host (e.g. using RPU) or rebooted.
• You must run secureboot-certs clear if you're updating from 1.2.0-2.4 or 1.2.0-3.1 and have previously run secureboot-certs install with the above versions installed.
• If you are unsure, it's recommended to run secureboot-certs clear anyway.
• If you have already propagated UEFI certificates to your existing VMs following the varstored 1.2.0-3.1 update, it's recommended to update to 1.2.0-3.2 and propagate UEFI certificates again.

@Greg_E said in XCP-ng 8.3 updates announcements and testing:

@dinhngtu

I was just getting ready to go update my pool... So is it safe now? And do these certs only affect Secure Boot VMs or UEFI VMs as well?

This problem will affect UEFI VMs. See the announcement above for more details.

In short, if you haven't updated, you should be safe; the offending update has been withdrawn and no longer available from yum update (as long as you're not using the testing branch).

@acebmxer said in XCP-ng 8.3 updates announcements and testing:

@dinhngtu Thank you for that reply.

Just to clarify I have 4 host. 2 host i updated to latest update once they became available to public via pool updates. The other 2 host I had to push the second beta release to get command secureboot-certs install to work. Then I pushed the rest of the update again once they became public.

So far im only seeing 1 vm having the issue on the later 2 host the had the beta updates installed. There that test PC and 1 windows server (Printsrver), xoproxy, and xoce.

Should i run those commands on all 4 host or just the later 2 with the troubled vm?

I haven't tested using different versions of varstored within the same pool.

@lukasz_s Hi, this is due to the withdrawn Secure Boot update.

You can recover by installing a preliminary test update here:

wget https://koji.xcp-ng.org/repos/user/8/8.3/xcpng-users.repo -O /etc/yum.repos.d/xcpng-users.repo
yum update --enablerepo=xcp-ng-ndinh1 varstored varstored-tools
secureboot-certs clear

Then propagate SB certs to make sure that subsequent dbx updates will run normally.

Sorry for the inconvenience.

@acebmxer Looks like it. You can recover by installing a preliminary test update here:

wget https://koji.xcp-ng.org/repos/user/8/8.3/xcpng-users.repo -O /etc/yum.repos.d/xcpng-users.repo
yum update --enablerepo=xcp-ng-ndinh1 varstored varstored-tools
secureboot-certs clear

Then propagate SB certs to make sure that subsequent dbx updates will run normally.

Sorry for the inconvenience.

@flakpyro It's in the middle. But the "d719b2cb-3d3a-4596-a3bc-dad00e67656f dbx" part is always the same across all VMs.

After reverting the updates and running secureboot-certs install again

This will create a pool-level dbx which does not have the problem seen in varstored-1.2.0-3.1. So as long as you propagated the variables to all affected VMs without any errors, there's no need to delete them manually. But you can always delete them again if you're not sure.

@plaidypus Could you configure your VM to collect a program crash dump (full dump)? https://learn.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps

@acebmxer You should have uninstalled VMware tools first before moving to other hypervisors, or the VMware tools will fail to uninstall. You can use the command given here to remove the tools after the fact: https://knowledge.broadcom.com/external/article/315639

@plaidypus I'm not sure why it's failing to report. Could you check the event logs?

@metheos I'll try to add it to the removal list. If the problem occurs again, could you run pnputil /enum-devices /disconnected /properties to see what's going on?

@manilx It's purely cosmetic and will clear on reboot.

@spelon Yes, it's known. I'll try to fix that in the next release.

@Greg_E Do you mean autoupdate? It's not planned just yet, but you can update them with Group Policy using our guide here: https://docs.xcp-ng.org/guides/winpv-update/ (I think you don't need the Autoreboot setting any more, but it's worth testing). You can also use other tools like SCCM, PDQ etc. The drivers themselves are production-ready, and should not cause the domain controller issue on Server 2025.

As for getting the drivers signed, the certificate did cost some time and money, but the most difficult part is dealing with Microsoft (since we're signing drivers, we needed not just a certificate but also a Microsoft hardware vendor account).