@gduperrey Updated both of my test hosts.
Machine 1:
Intel Xeon E-2336
SuperMicro board.
Machine 2:
Minisforum MS-01
i9-13900H (e-cores disabled)
32 GB Ram
Using Intel X710 onboard NIC
Everything rebooted and came up fine. Ran a backup job which also completed without issue. Using a mix of ext, XFS and NFS SRs
This seems to be fixed in the latest commit and also does not happen in the just released XOA 5.108.0
@florent i have NBD enabled. Whats strange is i updated builds twice last week and backups continued to run. (Though progress in the tasks view disappeared as pointed out in this thread).
Last night we had a storm that knocked power out and my UPS's ran out of juice taking my home lab down, when everything came back up after backups would no longer run and would show ""invalid HTTP header in response body". I rolled back to a earlier commit (c7167 June 5th) and now not only are backups working again but progress in the tasks view is also displaying properly again.
Edit: Moved to commit 7994f from June 23rd and i still see progress there as well.
Mine was working on the latest commit however tonight due to a power failure my XO VM rebooted and now i too am experiencing the same errors. My hosts also rebooted as a result. Rolled back to a build from earlier this month and all is well again.
@manilx I am seeing this as well (on 2effd)
Vmware Education pricing back before Broadcom was insanely good. We are getting an Edu discount from Vates as well but its still more expensive than pre Broadcom VMware was. At the same time it's much cheaper than Broadcom owned VMware 
I just recently created a new "Golden Template" of AlamLinux, i noticed AlmaLinux 9 used BIOS in the XO provided Template however Rocky Linux 9 uses UEFI. Is there any particular reason for this? Is there a performance difference between either? I'm wondering if i need to start over and select the Rocky Template if UEFI does perform better. I also notice Ubuntu uses UEFI but Debian uses BIOS however i also understand there is much more of a difference there.
@rtjdamen I do know the dedicated migration network was an issue. CBT data would be reset if you preformed a migration using a dedicated migration network, removing the dedicated network was a work around there. The next issue was taking a manual snapshot and removing would sometimes also reset CBT data. Perhaps i need to spin up a test VM and try again and i know there have been a lot of updates.
Not to dig up an old thread but was the issue with CBT being reset to 00000000-0000-0000-0000-000000000000 after a manual snapshot is taken with NFS storage ever confirmed or reproduced by Vates? I would love to start being able to use CBT and purge snapshots after each backup run, or have there been enough changes / improvements that this is worth testing again?
@dinhngtu here is the output from one of the VMs recently migrated:
xe vm-param-get uuid=261634d9-b67c-1048-b028-2e33abea6329 param-name=PV-drivers-version
micro: -1; xennet: XenServer 9.1.7.65 ; xeniface: XenServer 9.1.12.94 ; xenvif: XenServer 9.1.13.107 ; xenvbd: XenServer 9.1.9.82 ; xenbus: XenServer 9.1.11.115
One thing i've noticed since upgrading to tools version 9.4.1 is that the version installed will display properly in XOA up until the VM is migrated. After a migration it changed to just "Management agent detected" with no version shown. Not sure if this is an XO issue or an issue with the tools itself?
@olivierlambert I think so! All my regular Delta+CR combination jobs run as usual tonight without issue which is good news after such a massive change.Ii can see the newly replicated copies when looking in XO-Lite or XCP-NG center so likely just a UI issue that is hopefully a quick and easy fix!
@florent so testing a CR job with the latest XOA release. The job runs successfully and i can see the virtual disk on the remote SR however the replica does not appear in XOAs VM inventory view.
Edits:
They do appear in XCP-NG Center,and XO Lite
They do NOT appear to show up using xo-cli list-objects
This also appears to ONLY happen to Windows VMs. Maybe related to missing templates?
VMs that have not yet even ran their daily replication job also seem affected. Replicas that ran last night and appeared in inventory before the update now now longer appear in the VM inventory list. This might be related to something else that was changed in the new XOA update?
@Tristis-Oris have CR jobs started working for you with the latest master code?
@florent Understod! Our XOA appliance has been on the latest branch since we deployed it almost a year and a half ago. Though with all these changes i may consider dropping it down to stable for a bit as backups are pretty critical. We only use Delta jobs (Which seem to be working well) and Continuous replication jobs. (Which i have seen some issues with recently in this thread).
Until the XO update is released by Vates you can filter your PV Driver version using the following:
pvDriversVersion:!9.4.1
This will show you all VMs not running that version. If you have Linux VMs they will be shown in this filter. If you have tagged your windows VMs you can combine this with a tag for example:
power_state:running pvDriversVersion:!9.4.1 tags:/^Windows$/
Remove the ! to filter for VMs that are running the latest 9.4.1 version.
So are we now at the point that Delta backups, Mirrors and CR backups are all working and considered stable? I have only been able to test delta backups with my home lab setup which have been working without missing a beat since the May 22nd fix for tasks not clearing. (MS-01 running XCP-NG, TrueNAS for storage)
We do use make use of CR jobs pretty extensively with the full XOA appliance for our DR site at work but because its full XOA and "production" i have not tested anything there. That said hoping for no nasty surprises with the may XO update if this is planned on being pushed to the stable XOA release!
@stormi Installed on our 2 production pools, DR and remote sites, 46 hosts total ranging from Dell, Lenovo, HP, and Supermicro servers, no issues to report!
I finally had some time to test the latest OME appliance you created. It booted up with version 4.3.x and it it appeared that i could in fact install plugins if i wanted! It prompted me to update to 4.4.1.2 which i did and which was sucessful! Sadly this looks to have overwrote whatever changes you had made as now after a reboot it will not let me install plugins any longer, and displays the usual error about disk space. The good news is that the guest tools and SSH access remain!
The OME VM is not configured with enough disk space to support this plugin. 400 GB of disk space must be allocated to the VM to support the appliance and all plugins (including this one). The VM has 0 GB allocated, 400 GB is required for the appliance and all other installed plugins
I attempted to edit the files you listed above, replacing references to /dev/sd?? to /dev/xvd?? and that seemed to fix it!
@Greg_E It seems to handle it fine, i have 2 of these systems running without issue with 8.3!