the writer IncrementalRemoteWriter has failed the step writer.beforeBackup() with error Lock file is already being held. It won't be used anymore in this job execution.

[manilx]

@olivierlambert I have it enabled all the time. And I'm now getting these errors now each day. Most of the time on the 1st backup of the day.

I get it on Delta, Mirror incremental backup

[manilx]

@manilx This has started 2 weeks go or so. But it's getting worse. I'm pretty sure it started with one of the XO updates..... As nothing else was changed.

[olivierlambert]

Are you on XOA or XO sources? Which version?

[manilx]

@olivierlambert XO Xen Orchestra, commit 987e9
Master, commit ef636

[olivierlambert]

I'm not sure to understand, why two different commit numbers?

[manilx]

@olivierlambert That's the last update I did yesterday.....

Just updated now to latest: Xen Orchestra, commit ef636
Master, commit ef636

[olivierlambert]

Do you have the issue on XOA?

[manilx]

@olivierlambert @Business with XOA no. Only HomeLab with XO but it has been rockstable until this.

[florent]

this happens when 2 backup jobs are working on the same VM , or a backup and a background merge worker .
If it's the first case, you can use a job sequence to ensure the backup run one after another

If it's the latter, you have a check box to ensure the merge is done directly inside the job (bottom of the advanced settings of the backup job ), instead of doing this in a background job. It will slow down a little the job, but ensure it's completely done.

[manilx]

@florent No other backup is running as I mentioned!

And "Merge backups synchronously" is on as mentioned.

This is not it.....

[florent]

@manilx are there multiple job running on the same VM ?

[manilx]

@florent Only one single backup job every 2hrs (multiple VM's). Running fore more than 1year without issues (not counting the initianl cbt stuff).

[manilx]

@florent This make no sense. There has to be some leftover sometimes of a lock file.

[SudoOracle]

So I started getting this a few days ago as well. My backups have been fine for over a year until now, they are constantly failing at random. They will usually succeed just fine if I restart the failed backup. I am currently on commit 494ea. I haven't had time to delve into this though. I just found this post when initially googling.

[olivierlambert]

That's interesting feedback, maybe a recently introduced bug. Could you try to go on an older commit to see if you can identify the culprit?

[manilx]

@SudoOracle Thx for chiming in!

Glad to know I'm not alone.

[SudoOracle]

@olivierlambert I went back to my prior version, which was commit 04dd9. And the very next backup finished successfully. Not sure if this was a fluke, but I have backups going every 2 hours and this was the first to finish successfully today without me clicking the button to retry the backup. I'll report tomorrow on whether they continue to work or not.

[olivierlambert]

Okay great, you can use git bisect to easily find the culprit. Now we also need to find the common situation between you and @manilx : type of backup, advanced options enabled, type of BR, and so on to also understand more about this.

[manilx]

@olivierlambert I have changed my backup strategy because of that:

Before:

Now:

After this the problem did no longer occur.

[olivierlambert]

Can you sum up what you did? I admit I don't have enough time to investigate differences between 2 screenshots