Windows Server 2025 on XCP-ng
-
Hi Greg_E
Yea, that are exactly the same findings as mine
I think to disable the Service could be a workaround for the moment. 9.4.0 did not make a difference compared to 9.3.0.
We will start our new project with 2025 with workaround in place and monitor the situation when a new version become available where the issue may have been fixed.
I'm glad for any further comment on this topic when new informations become available from any the community members or the devs
Have a nice evening!
-
I'm trying to make this happen on my test VM. Are you saying your need to promote the VM to a domain controller to kick off this bug? I created my VM using the Server 2022 template which did not setup a vTPM or enable secure boot. I'm wondering if that's at all related?
-
Yes, you need to promote this to at least AD DS and go through the setup phases of this task. It was fine until I finished the AD set up and rebooted. I'm thinking there is a port that the new 2025 functional level uses that may be conflicting with the management agent. I didn't go too much farther because I need to move on with some other things in my lab.
If you disable that service, will the VM still be able to live migrate from one host to another during things like rolling pool reboot or rolling pool upgrade? Again, didn't have time to test right now, but have had issues in the past from this. Wish I remembered to try this while it was still built.
There may be one other way around this that might be worth testing... Build a Server 2022, set up AD DS and make sure everything is working. Then do an inplace upgrade to server 2025. This will keep the functional level at 2016 and check to see if everything is working. Then upgrade the functional level to 2025 and see what happens. Depending on where my other tests go, I might give this a try because I'm more likely to do an inplace upgrade on my production machines than to do a fresh install and migrate FSMO roles. But not right now.
-
@Greg_E
For the heck of it, I set up a new VM with 2025 and made it a domain controller complete with AD DS. It seems to work fine but, like yours, there are thirty something conhost processes running. -
This post is deleted! -
You should also notice that you cannot run any task scheduler tasks any longer. The task starts, but the action is not taken and after the timeout it end. Also any .msi package cannot be installed due to the Windows Installer Service is not able to start.
-
I noticed the installer when I was trying to update from the 9.3.3 to 9.4 agent, didn't know about schedulers being locked up.
-
@Chemikant784
So with this being patch Tuesday, two updates got installed:2024-11 Cumulative Update for Microsoft server operating system version 24H2 for x64-based Systems (KB5046617)
2024-11 Cumulative Update for .NET Framework 3.5 and 4.8.1 for Microsoft server operating system version 24H2 for x64 (KB5045934)For whatever reason, those processes are no longer showing up and I can run things from the scheduler??
If you updated tonight, did yours go away?
-
@archw
Hi, i thought the same today so i gave a try. But unfortunately at my test server fully patched with fresh ADDC on it there are the same issues present. But i have to say that i did not a detailed test at this time. -
The latest WS2025 patches fixed the issue for me.
-
Thanks, I'll have to give this a try when I get some time, I can set it up on the gigabit network on my lab. Might even do this today because it's a task I can do while getting interrupted.
-
Ok, this was again not as straight forward as we might like. System as follows:
XCP-NG 8.3 (current)
vTPM, vUEFI, vSecureboot
NFS share
Management Agent 9.4
Server 2025 with November 2024 cumulative update applied (not important)
AD DS, DNS, DHCP installed and configuredAfter several reboots I was still getting the problem:
Thinking a little more, I decided to switch from Automatic to Automatic (Delayed Start)
The delayed start seems to have fixed the issue for me:
I've had this issue before with other services, we handle audio and video devices which have some demanding start routines so I should have thought of this earlier. Simply changing to delayed start lets another necessary service start which then allows the MA to start.
Also, the horrible lagging I mentioned earlier was from a Truenas issue that I didn't know about until a couple days ago, patch your Electric Eel if you haven't done this yet. It's still a little laggy, but not like it was when I mentioned the problem before. All work so far has been through the XO console which is running on an old HP T630 which is pretty slow.
Going to monitor this for a while before I dig too deeply into setting this up, it's my second physical network on my lab which is also the second physical network on my VMware system, I'm sure I'll need to do more configuration than just DNS and DHCP down the road. I'll probably form a trust with the first network AD DS, just to give that some practice and see what 2025 might bring. Eventually I'll need to upgrade my production network to 2025, so I'm justifying the time spent in research for the future, I'm already Windows 11 for all my clients, so Server 2025 should play more nicely and might bring some features back that they removed from 2022 (some GPO to start).
-
@Greg_E
Hi Greg_EThanks for your detailed report and informations
I will try it with delayed start. It could be a solution indeed. When i have news on this, i will share it here. -
My three test systems seems to work properly when the management agent start type is set to delayed. So i think this can be a workaround for the moment.
-
Since it doesn't seem to hurt anything, maybe it should get pushed up the chain to Xenserver and have them just change the MA installer to make this delayed start. But for now maybe this needs to be a note in the documentation that the user may need to change to delayed start for Windows 2025 running any parts of AD. If there are clearer images needed, let us know and one of us will grab some. Fell free to use my images in this thread in the official documentation if desired/needed. Should probably work up a few showing what this looks like from "normal" server view (no GUI), because as we know (people keep telling me) real admins don't use a GUI!
(aside, if that's true, why does Xen Orchestra exist??? Why does vCenter??? Why does Prism???
)Also keep an eye on your server 2022, you never know when Microsoft will push a "fix" for something in AD and break the MA on currently working systems. We all know they have a habit of doing things like this. But I have a feeling it is the functional level of 2025 that is breaking this, something that should probably be tested but I don't have the time right now to delete it and set it back up at functional level 2016 and see what happens.
-
Also I should say that I've rebooted many times and still OK, going to leave it baking while I get on to other things.
-
After some testing, I've found that running any service that creates a form (of which Xen management agent is one) will trigger the issue. This points to an issue in Windows Server itself rather than the Xen guest agent.I've created a thread on Microsoft Community to keep track of the issue: https://answers.microsoft.com/en-us/windowserver/forum/server_performance-servertop_application/creating-an-ui-autostart-service-on-a-server-2025/c2ced91b-21c2-49c1-86a6-24b90dc72c6d. In the meantime, please use @Greg_E's method of setting the Xen management agent service to delayed start to work around this issue.I'm investigating further.
-
I haven't had time to install anything else on this VM, I'll have to think about other roles or features that I might want to try to see if additional services will cause additional problems. WDS might be a good one since it opens up a few different services, but really only good for Win10 stuff these days.
I could install the Shoutcast server (as a service), but not sure if it plays nice with AD DS, would be simple enough to point a stream at it and play the audio locally once it is set up.
Do you think Zabbix agent would trigger it? I could get that going pretty quick and use it to test things with Zabbix while doing it.
-
Another weird thing I've noticed with Windows Server 2025 on XCP-ng.
The network keeps resetting itself to "Public" (vs domain). This only happens when the VM is a domain controller and it only happens with Server 2025. If you go into the VM's console and disable the NIC and then re-enable it, it returns to a domain network. I've tried the usual trick of change the "Network location awareness" to delayed start but it doesn't help.
-
@archw Yes,
I'm seeing the network changing too. But no rhyme or reason behind it. I know I changed this once last week, and had to change it this morning. Did a reboot and it is still OK so I'll have to monitor this as well.
So far after moving the MA service to delayed start, things are working fine in this regard. The only service I've added is the Zabbix Agent2 and connected it to my Zabbix server. So far, so good.
I'm going to say that I won't be upgrading any of my production servers to 2025 until summer of 2025, just to be safe and hope all these little oddities get worked out. I'm guessing most of these are Windows changes that don't really play nice with the PV drivers. Have to see how quickly these drivers get updated.