Mirror of full backups with low retention - copies all vms and then deletes them

Forza

I noticed that for "Mirror full backup", the process copies every backup, and then immediately removes backups older than retention. This seems unnecesary for full backups as there is no relationship between each backup for a VM.

Pilow

@Forza I can confirm the behavior, all points being copied, then pruned to target retention...

for now it seems to not be a bug but a feature, to not have to think of what happens in the source job in smart mode (VM added/substracted to the source job)

"we copy all and see afterward"

but @bastien-nollet and @florent can confirm on that ?

ps: I had this on mirror incremental backup, as I do not do full backups.

Forza

It looks like it transfers one backup, deletes it, starts the next backup, deletes it, starts the next one... and so on. This seems rather inefficient for full backups. I can understand it has to transfer the full chain when dealing with incremental backups, even if it has to prune and merge them afterwards.

I also notice that even though I set the retention to 1000 in the full mirror job, not all backups are copied:

    {
      "data": {
        "type": "VM",
        "id": "0ecd9bc3-b4e8-8f0e-e50d-6b94420ea742"
      },
      "id": "1764584306124",
      "message": "backup VM",
      "start": 1764584306124,
      "status": "success",
      "infos": [
        {
          "message": "No new data to upload for this VM"
        },
        {
          "message": "No healthCheck needed because no data was transferred."
        }
      ],
      "tasks": [
        {
          "id": "1764584306137:1",
          "message": "clean-vm",
          "start": 1764584306137,
          "status": "success",
          "end": 1764584306150,
          "result": {
            "merge": false
          }
        }
      ],
      "end": 1764584306151
    },

Bastien Nollet

I confirm that this is the current behaviour, as @pilow reported here https://xcp-ng.org/forum/post/99446

We might change it in the future to make it better, but it won't be trivial to change.