@DevilDan Windows 11 never fails to surprise... Can you try with Server 2025?

There's a specific timing needed to attach the debugger, refer to this thread: https://xcp-ng.org/forum/topic/10284/vm-failing-to-reboot

@DevilDan It's only mandatory for the installation. Once it's installed you can disable secure boot and enable debugging.

@DevilDan I noticed that in your earlier freezes, the CPU went to 0% but in these recent freezes the CPU is pegged to 100% instead. Does your guest still behave the same way during the freeze (no RDP/XOA console but still responding to ping)? It's an advanced process, but you can try attaching a kernel debugger and see what's happening. https://docs.xcp-ng.org/troubleshooting/windows-pv-tools/#connecting-to-guests-using-serial-console

@DevilDan Could you confirm that it's UEFI mode that's causing the problem?

@Forza The link you posted looks to be the same issue, just from a different kernel version.

@DevilDan If the problem shows up on older Windows versions I would start suspecting a few other factors.

• Do you have any Windows PV drivers installed?
• Could you switch back to the Realtek NIC model instead of Intel e1000?
• Does it happen on all VMs at once or just one VM at a time?
• Any issues with Linux distros?

@DevilDan Which software was uninstalled if you don't mind telling?

@guiand888 Thank you for the feedback. Just a note: before running XenClean, please turn off the "Manage Citrix PV drivers via Windows Update" option; otherwise, Windows will try to reinstall Citrix drivers as soon as it's rebooted.

For the storage perf, Xen PV does fluctuate a lot at I/O performance (both network and storage), so the real performance would be better seen with long benchmark runs (depending on your I/O backend).

@guiand888 Thanks very much for testing. Did the IP/guest version reporting in XO work for you? The incompatible drivers warning is a bit unexpected, I'll try to fix that.

For the disk performance, did you install with the Xen PV disk drivers option enabled? If yes, it might just be per-test variance (which happens a lot with Xen PV drivers), as the sequential write speed differed a lot between 9000.0 and 9030/Citrix guest tools.

@Greg_E With which file and AV are you getting a detection? I'll submit a false positive report.

And yes, getting drivers signed by Microsoft has been a massive issue that I (and other Windows software vendors) are encountering.

I've uploaded an ISO file to the release. You could also generate one yourself with WinCDEmu or a similar tool. (I deleted the ZIP file by accident and had to reupload, the hashes should be the same)

Please see the answers below:

Will this installer remove the Xenserver drivers if they are already installed? Or should I use the cleaner first?

No, the installer will not remove existing XenServer drivers. You must use the included XenClean tools first.

The instructions say to remove secure boot, then run a powershell script, isn't this going to cause a problem with Windows 11 and maybe Server 2025? Can secure boot be turned back on after the powershell script has been run?

No, since our drivers are testsigned. Running Windows in testsigned mode should pose no problems on a non-production system.

I'm trying to slice out some time to work with Server 2025 in my XCP-ng lab so I can move forward in my production system when I'm comfortable with the upgrade. I'm on 8.3 stable if that matters, I could move to 8.3 testing if needed for these drivers/management agent. I'd really like to move away from Xenserver drivers because I just don't trust them to remain available in the long term.

The drivers are tested on 8.2 and 8.3, you shouldn't need to upgrade your hosts.

I'm also trying to keep my Server 2025 in UEFI, Secureboot, and vTPM going forward. We are going to be forced sooner or later so I might as well get familiar with any problems that might appear now while I can test.

We're trying our best to get our drivers signed by Microsoft for Secure Boot compatibility as soon as possible. Please stay tuned for updates.

@DevilDan I do know that Server 2025 has some issues has well. Probably best to avoid the version for now.

@DevilDan Could you also describe your host hardware and special configuration (if any)? Does it happen on other operating systems or just Windows 11?

@DevilDan What if you uninstalled all Xen tools?

edit: Before using XenClean or XenBootFix, please read carefully the instructions on the XCP-ng documentation website!

Hello all,

Version 9.0.9030 of the new Windows PV drivers has been released.

This release brings multiple driver stability fixes, a new Rust-based Xen Guest Agent and the XenBootFix boot repair tool.

This driver release is for testing purposes only and not for production use. It requires putting the system into testsign mode. (only applies to the drivers themselves, not XenBootFix and XenClean; you can use the two tools separately without any configuration)

Download the release here: https://github.com/xcp-ng/win-pv-drivers/releases/tag/v9.0.9030

Release notes are below:

Before Installing

• This driver release is for testing purposes only. Not for production use!
• Make backups/snapshots before installing!
• The drivers in this package are testsigned and require enabling testsigning mode. Disable Secure Boot, then run the included script testsign/install.ps1 as Administrator to configure Windows and install the necessary signer certificates. Your VM will reboot automatically.
• If you're running version 9.0.9000.0, we're interested in hearing about your upgrade experience to 9.0.9030.

Changes since 9.0.9000.0

• NEW: Includes the new Rust-based Xen Guest Agent for resource and IP address reporting.
• NEW: Includes the new XenBootFix boot repair tool for VMs rendered unbootable by any Xen drivers (Xen PV, XCP-ng, Citrix, etc.)
• NOTICE: Xen PV disk drivers are now disabled by default on new installations. The default emulated NVMe driver will be used instead. Existing installations will not be affected.
• Fixes: Many more installation, uninstallation and upgrade safety checks are now built into the installer.
• Fixes: Various improvements to XenClean for more complete removal of driver services.
• Fixes: xenvbd: Xen PV disk drives now report as SSD drives to avoid unnecessary defragmentation (commit)
• Fixes: xenvif: Fix various VM hanging and crashing issues when changing VM network in Xen Orchestra (commit 1, commit 2)
• Debug symbols are now included with the installation package.

Changes since 8.2.2-beta

• This release is based on upstream 9.1-series drivers, which includes the PV Mouse/Keyboard driver and PV Console Driver. You can access the PV console with the following command from a XCP-ng host:

xl console -t pv <vmname>

• New installer with clean uninstallation and multiple install/uninstall safety checks.
• New XenClean utility for cleanly removing XCP-ng and Citrix drivers.
• Multiple driver stability fixes.
• Volume Shadow Service Provider is no longer included. (XCP-ng 8.1 and newer no longer support quiescent snapshots)
• Management Agent is currently not included. Xen Orchestra may show "Management agent not detected", but shutdown and reboot features will work normally. We're looking to bring back features of the management agent in the next releases. Check out the new Rust-based Xen Guest Agent included in our package! We're looking to bring back even more features to the agent.
• Older Windows versions are no longer supported. The driver requires at a minimum Windows 10 1607 or Windows Server 2016.

Help / Community Support

If you encounter installation/uninstallation errors, please try again with one of the following commands:

For installing:
msiexec.exe /i XenDrivers-x64.msi /l*vx install.log

For uninstalling:
msiexec.exe /x XenDrivers-x64.msi /l*vx uninstall.log

Please include this log along with the file C:\Windows\INF\setupapi.dev.log in your bug report. These files will help us troubleshoot any installation issues.

Additionally, please report any errors or BSOD you encounter during testing of this release. Your feedback is very appreciated.

• Discussion: https://xcp-ng.org/forum (preferred)
• Issue Tracker: https://github.com/xcp-ng/xcp/issues
• IRC: #xcp-ng and #xcp-ng-dev on irc.freenode.net

Full Changelog: https://github.com/xcp-ng/win-pv-drivers/compare/v9.0.9000.0...v9.0.9030

@cds5296 Hi, could you list your hardware specs (server model, CPU model) and post a crashdump?

@yzgulec Could you try the following config, which worked for me:

URI: ldaps://<DC FQDN>
Copy ADCS root CA to /usr/local/share/ca-certificates/root.crt
Certificate Authorities: /usr/local/share/ca-certificates/root.crt
Check certificate: enabled
Base: OU=...
StartTLS: off
Credentials: fill
dn: <service account UPN>
password: <password>
User filter: (userPrincipalName={{name}})
ID attribute: sAMAccountName

@kagbasi-ngc Could you apply the hardening controls now and see if the VM still works?

@mrchip Could you post the contents of /var/log/{xensource,daemon}.log on the host (as well as their .1 version) as well? They're pretty big, so you'll have to upload them somewhere.

@mrchip Could you post a kernel log?