Ok so 1s is slightly not enough, thanks for the update.

Thanks again @Pilow

I don't think the remotes being S3 changes something here.

@cbaguzman for information, I made some changes on vhd-cli so in the future we'll get a more explanatory error message when a command failed because we passed an incorrect argument: https://github.com/vatesfr/xen-orchestra/pull/9386

b-Nollet opened this pull request in vatesfr/xen-orchestra

closed feat(vhd-cli): prevent using invalid options #9386

Hi @Pilow,

I've done some more testing and looked at the code, and I wasn't able to reproduce this behaviour once. It's also unclear to me why it can happen.

We may just add the delay as you did, but 10s is probably too long. Could you try to replace it by a 1s delay instead, and tell us if it's enough?

Hi @cbaguzman,

I tested on my own and I got the same result as you, but then I realized the AI you used both tricked us into thinking that the --chain was a valid option for the info command (it's not).

I removed this option and the command worked properly.
Can you try the same command without this option?

Hi @Pilow,

Thanks again for the feedback, I think now we have enough data to be sure it's indeed a race condition.

We noticed that the log you sent earlier in this topic is a backup job using a proxy. Could you tell if the backup jobs that ended up with a wrong status in the report were all using a proxy, or not all of them?

Thanks @Pilow for the report, I'll try to reproduce on my side to get a better undertanding of what's creates the fallback.

Thanks @Pilow for the tests.

We'll have to investigate this to fix it more properly than adding a ugly delay.

I agree, let's wait for more runs.

If it's indeed a race condition, we'll still have to figure out a better way to settle this than just adding delay

Hi @pilow,

Currently, I don't know what would cause this or why this would happen more frequently.

Could you test on your own to add some delay before sending the report, to see if it's indeed a race condition?

To do that, you just need to edit the file packages/xo-server-backup-reports/dist/index.js by adding these two lines:

const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
await delay(10000);

at the beginning of the _report function, like this:

  async _report(runJobId, {
    type
  } = {}, force) {
    const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
    await delay(10000);
    if (type === 'call') {
      return;
    }

then just restart xo-server.

We just merged a PR that changes the UI to allow offline backups for delta backups, disaster recovery and continuous replication: https://github.com/vatesfr/xen-orchestra/pull/9228

Now, if your delta backup makes offline backups, you can disable that without having to change the backup type.

b-Nollet opened this pull request in vatesfr/xen-orchestra

closed fix(xo-web): allow offline delta backups #9228

Hi @Forza,

I made a fix to prevent the "No new data to upload for this VM" message from appearing if it is only true for a part of the backup jobs of one VM.

It's available here: https://github.com/vatesfr/xen-orchestra/pull/9286

You can test it by switching branch if you're running XO from sources, otherwise you'll need to wait for the next end of month release.

b-Nollet opened this pull request in vatesfr/xen-orchestra

closed fix(backups): fix incorrect info on mirror backups #9286

@Forza

Thanks for the details, I managed to reproduce after a few trials.

I'll try to fix it and keep you informed.

@Forza

As the named are greyed-out, could you confirm that the delta backup job saves the backup to remotes srv04-incremental and srv12-incremental, and the mirror backup job copies the backups from srv12-incremental to srv04-incremental? (or at least confirm these are the same couple of remotes)

Also, could you tell me if the VM with which you got the "No new data to upload for this VM" message was backed up by your backup job "Incremental backup every 4 hours - 8 days retention"?

Thanks @Forza,
I'll try on my own with a similar configuration.

I confirm that this is the current behaviour, as @pilow reported here https://xcp-ng.org/forum/post/99446

We might change it in the future to make it better, but it won't be trivial to change.

Hi @Forza,

Could you send us a screenshot of your backup job configuration so we can try to reproduce?

Hi @k11maris,

I think the situation you showed in your screenshot where both options are selected but are both greyed out should not happen.

We will try to reproduce it on our side and fix it.

Hi @cbaguzman,

We just merged a PR that should allow you to use vhd-cli check on encrypted remotes: https://github.com/vatesfr/xen-orchestra/pull/9235

Please note that, as stated in the PR description, this PR changes the parameters you need to give to several vhd-cli commands, including the parameters of vhd-cli check. You can use vhd-cli check --help for details.

Here's an example of a command I ran on an encrypted remote:

vhd-cli check --chain 'file:///localRemoteEncryptedTEST?useVhdDirectory=true&encryptionKey=%2219876543210987654321098765432109%22' 'xo-vm-backups/695550e5-16e4-4728-6669-ac39e79f19e2/vdis/2656e4f4-efbc-47a6-95aa-0ba88632e545/7a1b144e-1ec8-44e8-9f5e-2d39119a228d/20251125T095729Z.alias.vhd'

b-Nollet opened this pull request in vatesfr/xen-orchestra

closed feat(vhd-cli): accept all types of remotes #9235

Hi @k11maris,

You can either specify the snapshot mode or check the "Offline backup" option.

It's a bit counter-intuitive, but to uncheck it, you can set "Snapshot mode" to "Normal", and you should be able to modify it. (then you can put back the snapshot mode to offline)