installed on 2 test machines
Machine 1:
Intel Xeon E-2336
SuperMicro board.
Machine 2:
Minisforum MS-01
i9-13900H
32 GB Ram
Using Intel X710 onboard NIC
Both machines installed fine and all VMs came up without issue after. My one test backup job also seemed to run without any issues.
@gduperrey installed on 2 test machines
Machine 1:
Intel Xeon E-2336
SuperMicro board.
Machine 2:
Minisforum MS-01
i9-13900H
32 GB Ram
Using Intel X710 onboard NIC
Both machines installed fine and all VMs came up without issue after.
I ran a backup job after to test snapshot coalesce, no issues there.
@stormi Updated a test machine running only couple VMs. Everything installed fine and rebooted without issue.
Machine is:
Intel Xeon E-2336
SuperMicro board.
One VM happens to be windows based with an Nvidia GPU passed though to it running Blue Iris using the MSR fixed found elsewhere on these forums, fix continues to work with this version of Xen. 
@dthenot @olivierlambert thanks guys ill hold off on submitting a ticket for now to keep the conversation centralized here but if you need any more info, would like me to try anything or would like a remote support tunnel opened just let me know! 
@CodeMercenary I am using V4 on the XO-Server to our backup remotes and it seems to work just fine. However using V4 as a storage SR was nothing but problems, as @nikade mentioned we had tons of NFS Server not responding issues which would lock up hosts and VMs causing downtime. Since moving to v3 that hasn't happened.
Checking a host's NFS retransmissions stats after 9 days of uptime i see we have had some retransmissions but they have not caused any downtime or even any timeout messages to appear in dmesg on the host.
[xcpng-prd-02 ~]# nfsstat -rc
Client rpc stats:
calls retrans authrefrsh
268513028 169 268537542
From what a gather from this blog post from redhat (https://www.redhat.com/sysadmin/using-nfsstat-nfsiostat) it seems like that amount of retransmissions is VERY low and not an issue.
@stormi installed on the same test machines i have the other batch of updates installed on. No issues after a reboot.
@gduperrey Installed on the same 2 hosts as the last batch of test updates released in December.
No issues to report so far, ran a backup job after after without issue.
@gduperrey Installed on a 2 host AMD based test pool, as well as our 5 host Intel based production pool without issue using rolling pool update. Everything migrated, updated then migrated back perfectly.
Also installed on my home server without issue.
@bleader Updated a small test server without any issues.
@olivierlambert Makes sense! I would schedule the replication to occur a couple hours after the backup runs are complete to ensure its a replica of all data and not a partial replica!
So i already have a ticket in for this but thought i'd post here to draw on the knowledge of the community as well.
We have a 5 host pool with several backup and replication jobs configured in XOA. Randomly a job will stall out with some VMs partially backed up, (the estimated end will shoot up to 2 - 3 days, and progress will stop) the only way i am able to cancel it is to restart the tool stack on the pool master and restart the XOA service on the XOA appliance. Then let GC run and retry the failed VMs. This usually then results in a full of the failed VM. This can happen to both CR jobs and to regular backup jobs.
In the log i see in XOA for the failed job is:
"message": "HTTP connection has timed out",
"name": "Error",
"stack": "Error: HTTP connection has timed out\n at ClientRequest.<anonymous> (/usr/local/lib/node_modules/xo-server/node_modules/http-request-plus/index.js:61:25)\n at ClientRequest.emit (node:events:518:28)\n at ClientRequest.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:52:17)\n at TLSSocket.emitRequestTimeout (node:_http_client:849:9)\n at Object.onceWrapper (node:events:632:28)\n at TLSSocket.emit (node:events:530:35)\n at TLSSocket.patchedEmit [as emit] (/usr/local/lib/node_modules/xo-server/node_modules/@xen-orchestra/log/configure.js:52:17)\n at Socket._onTimeout (node:net:595:8)\n at listOnTimeout (node:internal/timers:581:17)\n at process.processTimers (node:internal/timers:519:7)"
}
Now i understand i can increase the timeout value, and that is what support has suggested as well as the XO documentation here:
https://docs.xen-orchestra.com/backup_troubleshooting#error-http-connection-has-timed-out
What i can't figure out is why this timeout would occur in the first place?
I have ensured the XOA appliance is running on the Pool master at all times, i have also ensured the pool master is the least loaded of all 5 hosts in our production pool in an attempt to mitigate this in the past with no success. The pool master is only running 17 VMs, has 32 cores and 384GB of RAM with 8 GB assigned to dom0.
Each host has 4x 10GBe SFP+ ports, 2 x 10GB Ports are in an multi-chasis LAG (LACP) and dedicated to storage (NFSv3), the other 2 are in another multi-chasis lag (LACP) and dedicated to VM traffic/Management/Backup traffic.
So i don't think anything is overloaded that would cause this? Any suggestions or something to look for? I have increased the timeout value for now but the docs seem to imply i need to get to the bottom of what is causing it for a more long term solution.
@stormi installed on the same test machines i have the other batch of updates installed on. No issues after a reboot.
@stormi Updated both of my test hosts. Everything rebooted and came up fine.
No VM stats in XO / XOA i see still. I will be curious if this round of updates fixes my EFI / Windows Server reboot hangs.
@Greg_E I do not have anything older like a K620, i do have a GeForce GTX 1650 in another machine that is also doing this but i believe that's the same generation as the T1000 (Turing). I agree i think ARC could be a great replacement for this application in the future.
Since this is occurring at boot before the OS loads i'm not sure if this would be management agent related?
The VM is EUFI with no vTPM or Secure Boot enabled. The CPU us a "Xeon E-2336 CPU @ 2.90GHz" Running in a Super Micro Server. We use these servers at remote sites to run a number of VMs including a "Blue Iris" server with an Nvidia T1000 GPU passed thru to it, i have one such servers as a test server as well. The second machine doing it is a Minisforum MS-01 also with a T1000 GPU Passed thru in my home lab. The OS in both cases is Windows Server 2025. Over the weekend i cloned a fresh copy without GPU Passthru to see if it occurs with no GPU. We have about 15 of these VMs running at remote locations on the stable 8.3 patch branch that are not doing this.
It should be noted though that these VMs did need to be customized to allow Blue Iris to run without BSODing the VM on Intel CPUs. Thread can be found here: https://xcp-ng.org/forum/topic/8873/windows-blue-iris-xcp-ng-8-3/35?_=1746455850378 but in the end the following needed to be applied to a VM to keep it from BSODing when running Blue Iris on new Intel CPUs:
xe vm-param-add uuid=... param-name=platform msr-relaxed=true
@stormi It does not seem to do it every time, it seems the VM must run for sometime and then be rebooted to cause it to happen. I have created a bug report file from the host as requested and will DM you a link to it! The last time i experienced this would have been at "May 2, 2025 at 4:18 PM (3 days ago)" according to our XOA appliance.
This happened again today when rebooting a Windows Server on my test host running these updates:
A "Force Reboot" does not solve it, i have to do a "Force Shutdown" and power on again. It will hang at that screen indefinitely. I can reproduce this on two hosts now. It seems if i reboot again immediately after the fact it will be fine but after it has sat running awhile it will always occur. Not sure if it may be GPU passthru releated. I can clone another Windows VM that has no passthru device enabled if that would help.
After running this for a week now i have come across an issue with EFI hosts with these testing updates. Sometimes when booting a VM or rebooting after a windows update the VM will hang at "Guest has not initialized display yet" and not boot. Doing a force shutdown and powering on again solves it. It has happened twice so far to unrelated windows VMs both in my homelab and in our test lab at work. Not sure how to provide more info or logs when it happens however.
Edit: One common thing between both these VMs is they both are using GPU Passthru, if that could at all matter.
@AtaxyaNetwork are you able to list the files that require these drive name modifications? I can try changing them on the latest updated version of the appliance.
@AtaxyaNetwork I understand it would be a lot of work to make a new XVA every time Dell updates the appliance. Would it be possible to write a shell script that we could copy to the appliance and run after the fact that could change the /dev/sda to /dev/xvda in the needed files perhaps? I can try running it on my updated appliance you sent me, im just not sure where the drives are being referenced.
Something like
sed -i -e 's/sda/xvda/g' /etc/whatever.conf?
So testing the appliance and here is what i have encountered so far:
Boots up no problem, XO sees tools installed and all stats work.
It took me two attempts to upgrade it to 4.4.0.75, the first attempt failed, i reverted a snapshot from before trying and upon trying again it was successful.
However i can no longer install / update plugins, back to throwing disk space capacity errors. I'm guessing whatever was modified in the appliance you shared was reverted by the update which now causes it to think there is no disk space again.