@CodeMercenary i have seen this as well, one of our backup repos was almost out of space, this decreased the backup speed, we thought of issues on the xoa side but it was just related to the repo itself…. So it is good to check both. The 1 gig part makes sense.

Excellent Happy to see our fix solved your problem and enhanced our compatibility with older XenServer version. Don't forget to upgrade, the path to get to XCP-ng is relatively straightforward

Is it PV, HVM or a PVHVM VM? Health Check can't work on HVM because there are no tools.

@cirosantos0 Indeed, our detection is wrong in this case, I will put this in our backlog, but don't expect it to be fixed soon. PRs are welcome though

An update, if anyone ever comes across this via search engine. Turns out it was my container's timezone. The image was set to pure UTC, no timezone, by default, so I believe when it was writing files to my network storage it introduced a discrepancy. My network share was recording the file metadata accurately to real-time, so I assume when it came time to do another backup, the file time XO expected was different, making it think it was "stale" or still being "held". Have now run both scheduled metadata and VM backups without any errors . In summary: make sure your time, date and timezone are set correctly!

@florent It was working before, this is likely something that has been broken when the handling of snapshots has been moved from VMs to VDIs. We need to fix this.

@probain Noted in backlog

Hi, That's because our doc isn't fully up to date with the new naming (even XO in the app itself). See this recap: [image: Schema-new-wording-backup-2.png] More details in here: https://xen-orchestra.com/blog/xen-orchestra-5-83/#backup-nomenclature

I don't think it's a good approach, BR weren't meant for that. If you want to send them offsite, mirror backup is the best option. About immutability, we have an external program doing it if you want.

@stephane-m-dev understood. My suggestion regarding the different behavior (does not work with disk share, does work with user share) on unraid would be to put this in a "tip" in the documentation section about setting up Remotes.

I re-checked again but the issue is unfortunately not resolved. It does not happen on all VMs and files, so maybe there is something wrong somehow in the VDI?

@McHenry Base copies are not copy per say but a base of a VDI chain. [image: beforemerge.png] Here you can see that A and B are base copies for the snapshots C and E while D is the active VDI. You can find more info on this page: https://docs.xcp-ng.org/storage/#-coalesce

that a make a sense, now. So no issue here.

@daniel-grimm definitely not related to cloud in my case!

@DavidMrLane said in Where to find the health-check results: The tasks log doesn't tell you which VM was health checked. If you click the Open raw log button, it will give you the details on the request, including the UUID of the VM being tested -- { "id": "0m2dmjnnh", "properties": { "name": "VM Backup Health Check", "objectId": "remote-2//xo-vm-backups/92832a7e-e72d-2d02-6a25-9802f1c24072/20241017T050741Z.json", "userId": "3b480e98-af4e-4dbf-9f1e-29ac59d6364f", "type": "backup.vm.healthCheck" }, "start": 1729189103741, "status": "success", "updatedAt": 1729189318550, "end": 1729189318550 } In this case, the VM's UUID is 92832a7e-e72d-2d02-6a25-9802f1c24072, and you can paste this value into the filter when viewing the VM list in XO to find the exact VM involved.

@planedrop said in Backup / Migration Performance: @KPS Regarding the 2TiB limitation, it'll definitely be nice when we have SMAPIv3 so we can go over this, but it's worth noting that IMO no VMs should be larger than this anyway. Generally speaking if you need that kind of space it'd be better to just use a NAS/iSCSI setup. Something like TrueNAS can delivery that at high speed, and then handle it's own backups and replication of it. I know most probably already know this, and all environments are different (I manage one that requires a 7TiB local disk, at least for the time being, plan is to migrate it to a NAS once the software vendor supports it), but it's worth noting anytime I see the 2TiB limit come up, ideally it should be architected around so the VMs are nimble. I do something similar w/ a pretty massive SMB share and TrueNAS can back this up at whatever speed the WAN can handle, in my case 2 gigabits and it'll maintain that 2 gigabit upload for 8+ hours without slowing down. (and I'm confident even 10 gigabit would be possible with this box) We have 1 exception and that is for Windows file servers which is backing our DFS. Except from those we dont allow VM's larger than 1Tb and if they're that big we do not back them up because it usually breaks and cause all kinds of problems.

@frank-s Think of it like this... You take the original backup with snapshot on October 1st. You then have a differential backup on October 8th. That differential only writes the changes that make the state of the server on October 8th different (hence the term differential) from the state of the server as it was on the full snapshot on October 1st. Now you delete the snapshot of the full system state taken on October 1st, leaving only the October 8th differential. It's now October 15th and you want to restore something from the October 8th differential and you figure you'll just make a new full snapshot from October 15th and then try to apply the October 8th differential. However, this won't work because the differential from October 8th will be missing key reference points it will be looking for in order to overlay its changes onto the snapshot. What else on that server changed from October 1st to October 15th that isn't contained in the differential? What changes occurred from October 8th to October 15th that are newer than what is on the differential? You could, potentially, brick your entire system if the differential started overwriting changes it sees from its system state that are actually newer than the data it contains. Differentials are entirely dependent on the full snapshot on which they are based. Any new snapshot taken after their creation will be totally foreign to the differential... those differentials will be looking for very specific system states that existed at the time the original snapshot was created, and those will be completely different from the system state as it will appear on a new snapshot created after the differential.

@olivierlambert I've finally had time to sit down and do some more troubleshooting. It seems that the issue is somehow tied to the backup-jobs themselves. Deploying XOA and subsequently importing XO-config from my XOCE instance. I continued to see several issues between both instances. These issues pretty much went away, when I re-did the backup-jobs from scratch. And are now much more in-line with what I'm excepting to see. This was done on both XOA Stable, Latest and XOCE I am no longer able to repoduce this at all. So my guess is that a bug of some sort got introduced with this being a constantly updated from source instance. Marking as solved. As I don't believe there is much more to do at this time.

@stephane-m-dev In the schedule You can select force full backup