Thanks @Bastien-Nollet and @olivierlambert . I'm going to do some tests and write the results. Thank you so much.

@dnordmann the tunnel is closed. By the way the patch will be release tomorrow on latest, by the end of december on stable thank you all for your patience and your help identifying the root cause of this bug

@SethNY you could even enhance this VM_LIST = with TAGs on VM, so that you manage the selection directly in XOA

Yes, and then Xen will reduce the RAM if there's not enough available to boot a new VM (reducing down to dynamic min). Static means it can't be changed live, while dynamic can.

@olivierlambert Thank you for the reply. There was a thread on here about the iso not being mounted properly and I commented about having power off the vm and back and it will load the iso. This is specifically true with the 0.4.0 for linux. Image pulled from the XO Hub will not mount the ios if the vm is powered on. There is Yellow Explanation mark next to it. Power off vm and power on vm iso is ok. Reboot vm does not let the ios to work.

@MajorP93 aha, yea. Per disk concurrency is important too.

@hoehnp we're aiming at sharing a very first public version before end of year, but don't hope for it to be anything complete or stable

@Forza Cancelling a backup run is also on our roadmap ,as soon as we have done reworking the tasks to use xo-task on the full chain, and not an hybrid of various iteration of the tasks objects better tracking of the backup run is also on our roadmap , this is also linked to the task changes plus are some issues with the size on the remote before being able to do this In any case, both are on our backlog

@AtaxyaNetwork @Pilow @bvitnik Ok, knowing that the p2 partition was likely an old backup was exactly the breadcrumb that I needed. I confirmed via chroot and xsconsole that p2 was xcp-ng v8.2.1, and that p1 was v8.3. Then I was able to check the initrd files there. Namely I needed to see the updated dates and the kernel version. Thankfully, I saw initrd files on the backup that were the correct kernel and hadn't been disturbed. I then just cloned the initrd files from the backup partition to the p1 partition and BOOM. We booted right up. Much appreciated for the assistance, everyone. I'll still need to figure out why my disks are disconnecting, but that will be another post for another day.

There are no other jobs. I've now spun up a completely separate, fresh install of XCP-ng 8.3 to test the symptoms mentioned in the OP. Steps taken Installed XCP-ng 8.3 Text console over SSH xe host-disable xe host-evacuate (not needed yet of course since it's a brand-new install) yum update Reboot Text console over SSH again Created local ISO SR xe sr-create name-label="Local ISO" type=iso device-config:location=/opt/var/iso_repository device-config:legacy_mode=true content-type=iso cd /opt/var/iso_repository wget # ISO for Ubuntu Server xe sr-scan uuid=07dcbf24-761d-1332-9cd3-d7d67de1aa22 XO Lite New VM Booted from ISO, installed Server Text console to VM over SSH apt update/upgrade installed xe-guest-utilities Installed XO from source (ronivay script) XO Import ISO for Ubuntu Mate New VM Booted from ISO, installed Mate apt update/upgrade xe-guest-utilities New CR backup job Nightly VMs: 1 (Mate) Retention: 15 Full: every 7 Exact same behaviour. After first (full) CR job run, additional (incremental) CR job runs results in one more 'unhealthy VDI'. I've engaged in no other shenanigans. Plain vanilla XCP-ng and XO. There are only two VMs on this host, the XO from source VM, and a desktop OS VM which is the only target of the CR job. There are zero exceptions in SMlog. What do you need to see?

@acebmxer Great. These are some YAML basics. You should read more about it . Following AI instructions without understanding is not going to take you far.

Adding @florent and/or @Bastien-Nollet in the loop

Adding @gregoire in the loop

@Pilow alright, cool! that sounds pretty much like a standard solution, too much options or details usually just confuses the end users

@bazzacad great, glad it worked for you

We'd recently got a relevant experience regarding this weird renaming to uuids. We had one orchestra managing one pool. ISOs were in an ISO SR, with an nfs4 serving it underneath. All fine till then. We added one second pool to the orchestra. Just a single host by itself. One of the very next days we discovered that all names in the ISO SR had been replaced by uuids. Removing/readding the sr to the new pool helped temporarily. Usual names appeared again. But after a few more days, again uuids. Where uuids were appearing, we could not select anything from the dropdown list in the console's cdrom. The list per pool was unpopulated. We tried separate the shares by offering the new pool an nfs4 share from the NAS, actually sharing the same source dir. It did mount but now there was a uuid uniq constraint that was violated, so we could not see no files at all in this new SR. It would not be an illogical thought to have an 'iso sr' attached once to the orchestra and be offered by the orchestra to all managed pools, without uuids, without uniqs etc. There seems to be an unnecessary complication here I think.

Adding @clemencebx for review/ideas on this;

So I actually posted these findings to the ZFSBootMenu Discussion github: https://github.com/zbm-dev/zfsbootmenu/discussions/787 Looks like they have tested ZBM against KVM but not Xen. Can Xen not handle a kernel being kexec'd?

@olivierlambert I'd like to see replicated vms in netbox, perhaps not syncing their IP address to avoid conflict DR VM as well