Windows Server 2025 on XCP-ng
-
I noticed the installer when I was trying to update from the 9.3.3 to 9.4 agent, didn't know about schedulers being locked up.
-
@Chemikant784
So with this being patch Tuesday, two updates got installed:2024-11 Cumulative Update for Microsoft server operating system version 24H2 for x64-based Systems (KB5046617)
2024-11 Cumulative Update for .NET Framework 3.5 and 4.8.1 for Microsoft server operating system version 24H2 for x64 (KB5045934)For whatever reason, those processes are no longer showing up and I can run things from the scheduler??
If you updated tonight, did yours go away?
-
@archw
Hi, i thought the same today so i gave a try. But unfortunately at my test server fully patched with fresh ADDC on it there are the same issues present. But i have to say that i did not a detailed test at this time. -
The latest WS2025 patches fixed the issue for me.
-
Thanks, I'll have to give this a try when I get some time, I can set it up on the gigabit network on my lab. Might even do this today because it's a task I can do while getting interrupted.
-
Ok, this was again not as straight forward as we might like. System as follows:
XCP-NG 8.3 (current)
vTPM, vUEFI, vSecureboot
NFS share
Management Agent 9.4
Server 2025 with November 2024 cumulative update applied (not important)
AD DS, DNS, DHCP installed and configuredAfter several reboots I was still getting the problem:
Thinking a little more, I decided to switch from Automatic to Automatic (Delayed Start)
The delayed start seems to have fixed the issue for me:
I've had this issue before with other services, we handle audio and video devices which have some demanding start routines so I should have thought of this earlier. Simply changing to delayed start lets another necessary service start which then allows the MA to start.
Also, the horrible lagging I mentioned earlier was from a Truenas issue that I didn't know about until a couple days ago, patch your Electric Eel if you haven't done this yet. It's still a little laggy, but not like it was when I mentioned the problem before. All work so far has been through the XO console which is running on an old HP T630 which is pretty slow.
Going to monitor this for a while before I dig too deeply into setting this up, it's my second physical network on my lab which is also the second physical network on my VMware system, I'm sure I'll need to do more configuration than just DNS and DHCP down the road. I'll probably form a trust with the first network AD DS, just to give that some practice and see what 2025 might bring. Eventually I'll need to upgrade my production network to 2025, so I'm justifying the time spent in research for the future, I'm already Windows 11 for all my clients, so Server 2025 should play more nicely and might bring some features back that they removed from 2022 (some GPO to start).
-
@Greg_E
Hi Greg_EThanks for your detailed report and informations
I will try it with delayed start. It could be a solution indeed. When i have news on this, i will share it here. -
My three test systems seems to work properly when the management agent start type is set to delayed. So i think this can be a workaround for the moment.
-
Since it doesn't seem to hurt anything, maybe it should get pushed up the chain to Xenserver and have them just change the MA installer to make this delayed start. But for now maybe this needs to be a note in the documentation that the user may need to change to delayed start for Windows 2025 running any parts of AD. If there are clearer images needed, let us know and one of us will grab some. Fell free to use my images in this thread in the official documentation if desired/needed. Should probably work up a few showing what this looks like from "normal" server view (no GUI), because as we know (people keep telling me) real admins don't use a GUI!
(aside, if that's true, why does Xen Orchestra exist??? Why does vCenter??? Why does Prism???
)Also keep an eye on your server 2022, you never know when Microsoft will push a "fix" for something in AD and break the MA on currently working systems. We all know they have a habit of doing things like this. But I have a feeling it is the functional level of 2025 that is breaking this, something that should probably be tested but I don't have the time right now to delete it and set it back up at functional level 2016 and see what happens.
-
Also I should say that I've rebooted many times and still OK, going to leave it baking while I get on to other things.
-
After some testing, I've found that running any service that creates a form (of which Xen management agent is one) will trigger the issue. This points to an issue in Windows Server itself rather than the Xen guest agent.I've created a thread on Microsoft Community to keep track of the issue: https://answers.microsoft.com/en-us/windowserver/forum/server_performance-servertop_application/creating-an-ui-autostart-service-on-a-server-2025/c2ced91b-21c2-49c1-86a6-24b90dc72c6d. In the meantime, please use @Greg_E's method of setting the Xen management agent service to delayed start to work around this issue.I'm investigating further.
-
I haven't had time to install anything else on this VM, I'll have to think about other roles or features that I might want to try to see if additional services will cause additional problems. WDS might be a good one since it opens up a few different services, but really only good for Win10 stuff these days.
I could install the Shoutcast server (as a service), but not sure if it plays nice with AD DS, would be simple enough to point a stream at it and play the audio locally once it is set up.
Do you think Zabbix agent would trigger it? I could get that going pretty quick and use it to test things with Zabbix while doing it.
-
Another weird thing I've noticed with Windows Server 2025 on XCP-ng.
The network keeps resetting itself to "Public" (vs domain). This only happens when the VM is a domain controller and it only happens with Server 2025. If you go into the VM's console and disable the NIC and then re-enable it, it returns to a domain network. I've tried the usual trick of change the "Network location awareness" to delayed start but it doesn't help.
-
@archw Yes,
I'm seeing the network changing too. But no rhyme or reason behind it. I know I changed this once last week, and had to change it this morning. Did a reboot and it is still OK so I'll have to monitor this as well.
So far after moving the MA service to delayed start, things are working fine in this regard. The only service I've added is the Zabbix Agent2 and connected it to my Zabbix server. So far, so good.
I'm going to say that I won't be upgrading any of my production servers to 2025 until summer of 2025, just to be safe and hope all these little oddities get worked out. I'm guessing most of these are Windows changes that don't really play nice with the PV drivers. Have to see how quickly these drivers get updated.
-
So what's the overall consensus on on Server 2025 with XCP-NG? I see XenServer is claiming its fully supported now. Is delayed start on the management agent still required? Or only if using it as a AD domain controller?
-
I haven't done much testing with it, but delayed start doesn't seem to cause any negative side effects. If you have a large number of services, maybe this could become an issue, but I doubt it.
It was fully supported when I installed it (I think).
If you need it for production, it seems stable enough with the delayed start change. Not sure if other roles or features need this change so I would just be aware of it and proceed.
Also of note, this only delays the management agent from reporting back to the Xen system, it does not delay the drivers from loading. The only complication I could see is booting the VM and immediately trying to migrate it or shut it back down. Normally the MA doesn't do a lot of work so again, delayed is probably fine as I'm seeing delayed is faster than I can log in to look at it.
-
@Greg_E Good to know! We are mostly on server 2022 currently when it comes to windows, but anything new would probably be deployed on 2025 so was wondering what the status was.
-
Hi,
I agree to this conclusion at this time.
Greetings
-
I have some more to add to this after playing with inplace upgrades:
Any VM that seems to have had Xen Tools 9.3.3 installed at some point (even if upgraded to 9.4 since) will fail to upgrade hanging on first reboot during the process. If the VM was created once 9.4 was the current stable release this is not an issue and the upgrade will run.
To fix this we need to totally remove all traces of XenTools drivers from the system
Before upgrading:
Open a command prompt as Administrator
cd "C:\Program Files\XenServer\XenTools" Run: uninstall.exe purge verboseReboot
Confirm Xen Management agent is not running via Xen Orchestra Console
Proceed with Server 2025 inplace Upgrade.
Once into Server 2025 you will need to run to Xen Tools MSI to reinstall the tools. It will detect the management agent is still present even though it is not running and the drivers have been removed, run though the uninstall process, reboot then install a clean copy of the latest available version of the tools.
-
Thanks, that will probably save me an hour this summer if I decide to upgrade my production system. By then it might all be fixed.