A fix has been merged on master, now the LTR should properly pick the first backup of each day, week, month and year instead of the last one: https://github.com/vatesfr/xen-orchestra/pull/9180

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed fix(backups): long term retention taking the most recent values #9180

@Forza

We plan to make it configurable during the upcoming months.

@Pilow

This is supposed to be the first day of the year, of the month, etc... But we just noticed a bug that makes LTR keep backups from the end of month, end of week, etc.

A fix is on the way.

Hi @Forza,

The first list is the correct one.
For instance, a single backup can satisfy both the monthly and yearly criteria.

Hi @Pilow,

You're indeed supposed to have more backups kept. (probably 9, as the backup of the current week is already kept by the schedule's retention).

We'll try to reproduce the problem on our side to see what's going on and fix it, but it will be a bit long as testing it requires a lot of waiting.

Thanks for the feedback, it will help us improve the feature.

@Pilow I was replying to Forza, who has only one schedule with cron pattern 5 18 * * 0. I edited my message to make it clearer that I wasn't talking of ph7's backups.

What you did seems totally fine to me.
Could you detail a bit more what was unexpected in the results, and share a screenshot of your backup job retention/LTR configuration, so there is no misunderstanding?

@Forza As your backup tasks executes only once a week, the schedule retention is redundant with using "Number of weekly backups kept" of LTR.

If keeping the last 52 weekly backups of this job is what you want, I recommend you to set the schedule retention to 1, and the number of weekly backups kept to 52. This way, if you happen to execute this backup job manually (in addition to its schedule), it will not delete the oldest backups.

Hi @Pilow,

This is not possible at the moment, but it's a feature we plan to add in the following months.

@Forza said in Full backup - new long-retention options:

Thanks for the quick feedback. Does it mean that the schedule's own retention is also honored separately in addition to the LTR?

Yes, a backup is kept if it matches one of the retention criteria, either the schedule's retention or the LTR. (the backup is not duplicated, we just check for both criteria to know if we should keep the backup or not)

Hi @Forza,

The long term retention is independent of the retention of the schedule: even with a backup retention of 1, the long-term retention will keep some backups separately.
Note that the LTR applies to every schedule, so it may be preferable to use it with a single schedule.

The fix comes from @florent so all the kudos are for him

The fix has been released in XO 5.110

We'll try to make it available on 5.110 but we can't guarantee it for the moment (I think the PR Olivier linked is more of a temporary fix than a definitive solution).

If you feel comfortable with directly modifying the code of your XO, you can apply this fix just by commenting two lines in a file and restarting xo-server. The lines to comment are these ones : https://github.com/vatesfr/xen-orchestra/pull/8882/commits/51a543df25af1b0069e07f6d9ad2608ed3476e29

fbeauchamp opened this pull request in vatesfr/xen-orchestra

closed fix(backup) error footer1 !== footer2 #8882

@olivierlambert I think you're right, we just wait for the XAPI field PV_drivers_detected of the restored VM's guest metrics to be true.

@icompit Thank you.

It seems that the backup process fails to get the file lock on the backup directory. Have you modified anything on your remote recently?

Also, could you tell me if this is happening on all of this job's runs, all of your backup runs, or if it only happened once on this specific backup job execution?

@icompit The EEXIST error has appeared with @florent's recent work on backups. I think he's investigating this problem, but he his on vacations now, and will be back in two weeks.

If this causes too much trouble for you, for the moment I would recommend you to go back to a previous XO version.

I'm not familiar with the new "Unknown system error" you got. Could you give us the log of that backup job execution?

Hi @Forza,

This is not possible at the moment. The XO log retention is set to 20,000 entries.

About audit logs, we have a task planned for the following months to try to add retention configuration. It's a bit complicated as deleting old audit logs interferes with audit log chain verification, so I don't think we can expect this feature to be released before a few months.

@AlexQuorum

Nice to hear it

In the meantime I've made a bugfix which will soon be available so you can edit the monitors instead of recreating them.

Hi @AlexQuorum,

I think you are getting this error because you try to use the smart mode ("All running VMs" / "All running hosts" / "All SRs") but also specify elements in the "Virtual Machines" field. This field has two purposes: it either allows you to select VMs you want to monitor when the "All running VMs" option is off, or it allows you to select VMs you want not to be monitored if both the "All running VMs" and "Exclude VMs" options are on. (same thing for the "SRs" and "Hosts" fields in other monitors) If you're not in one of those cases, we don't allow values in this field to avoid confusion about the plugin behavior.

There is currently a bug on the plugin configuration that don't let users empty the "VMs" field without getting an error (it will be patched soon), so I recommend you to create new monitors with "All running VMs" on and "Virtual Machines" empty, and delete the monitors you previously created.

Hi @KPS ,

The difference between these two settings is that sessionCookieValidity determines the time before a user gets disconnected if they did not check the "Remember me" option, and permanentCookieValidity determines this when this option was checked.

If you want to force users to be disconnected after 12 hours regardless of how they connected, I think you need to set both sessionCookieValidity = '12 hours' and permanentCookieValidity = '12 hours'.

However, this memory increase you're experiencing is intriguing, it is not an intended behaviour.