<![CDATA[S3 Backup Retention Not Deleting Old Backups]]>Hi,

Been testing the S3 backup and it seems our b2 storage keeps growing even though the amount of data being backed up is pretty constant. I'm wondering if the S3 backup deletes old backups from b2 as part of the backup retention set in XO backup schedule? Or if I have setup something wrong on either XO or b2 which is causing the storage on b2 to keep growing without rotation.

Here is what I have as part of my backup:

125608b7-db1e-4c9f-99f6-c735729c7089-image.png
5e79fb78-24dc-498c-94bf-0145592079e8-image.png

f36a3fa2-9795-4976-a8bb-60e3ffdc7302-image.png

In b2, I have the following:
c23d0a5b-8110-4724-912f-2c8f1b886bf9-image.png

3fee8c20-80af-4bc9-9217-edbbd9e3d294-image.png

Thx,

SW

]]>https://xcp-ng.org/forum/topic/5432/s3-backup-retention-not-deleting-old-backupsRSS for NodeSun, 07 Jun 2026 02:15:27 GMTThu, 13 Jan 2022 00:39:42 GMT60<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Mon, 24 Jan 2022 14:59:21 GMT]]>Dunno it's @florent you should ask 😉

]]>https://xcp-ng.org/forum/post/46067https://xcp-ng.org/forum/post/46067Mon, 24 Jan 2022 14:59:21 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Mon, 24 Jan 2022 15:04:58 GMT]]>@olivierlambert, if an admin selects to delete backups, does XO delete these from the S3 bucket?

Last night, I selected to delete all backups from XO that are stored on b2 S3 bucket (about 50 backups). However when I checked b2 this morning, it seems nothing got deleted from the bucket:

2a4910de-2f4e-4271-9836-656da4a594b2-image.png

The above bucket contains only XO backups (for VMs and Pool/Config backups) so the bucket size should have reduced to only a few hundred MBs instead of still being over 10TB.

]]>https://xcp-ng.org/forum/post/46061https://xcp-ng.org/forum/post/46061Mon, 24 Jan 2022 15:04:58 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Mon, 24 Jan 2022 13:25:47 GMT]]>@andrew thank you for the clarification! XO isn't very clear on that as I thought after the 7 backup retention, it will automatically run a new full back. So do you think doing this will work where to offer 1 full backup per week and then 6 deltas in between:

78beb6d8-7119-42bc-96cc-84c17c308ed9-image.png

]]>https://xcp-ng.org/forum/post/46058https://xcp-ng.org/forum/post/46058Mon, 24 Jan 2022 13:25:47 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Mon, 24 Jan 2022 12:54:26 GMT]]>@stevewest15 Only running one full backup the first time and then deltas going forward is creating a synthetic full backup (deltas merged into the original full backup). The problem with this is if there is an error in the program (coding) or a delta backup error (or S3 error) then your "full" backup will be corrupted going forward, for ever... until the next real full backup. So you'll still want to run a true full backup regularly to make sure that you don't have long term errors in the synthetic full backup.

Other companies (like Veem) use a single full copy for both forward and reverse deltas to save space. For now XO only uses forward deltas so there will be more than one full backup at times.

@florent Testing today's XO master update, S3 backup works faster than ever. As this feature is still "beta" it's changing every update (normally improving).

]]>https://xcp-ng.org/forum/post/46057https://xcp-ng.org/forum/post/46057Mon, 24 Jan 2022 12:54:26 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Mon, 24 Jan 2022 03:07:11 GMT]]>@andrew Thanks Andrew! I just updated to latest commit and will run backup job again based on the following new settings (see below). I did remove the advanced settings of "Full backup interval" to blank so not sure if that matters or not when doing deltas:

eb0c0a84-f7dc-4bce-b904-57dfa830a74c-image.png

]]>https://xcp-ng.org/forum/post/46050https://xcp-ng.org/forum/post/46050Mon, 24 Jan 2022 03:07:11 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Mon, 24 Jan 2022 03:02:28 GMT]]>@stevewest15 If you are running XO from Source then you must update to the newest master. S3 backups just started working correctly for me on the newest version. If you are using XOA or another older tag version then it has old code for S3 and has issues with delta backups.

Second, your backup retention is only set to 3. If you want 7 restore points then you want to increase the retention.

With a delta backup you will always have at least 1 or 2 full backups saved. You will then have additional deltas between them to get a restore point.

The Full Backup interval does not determine how many full backups you keep but how often you have a full from which future deltas are based (and merged into).

Full+delta,,,,+delta = restore point.

The system will need a 2nd full backup as your deltas age and hit your Full backup interval and new deltas are based off of the new full backup. When all the old deltas are merged into the old full backup it will be removed and you will again only have one full backup plus deltas.

]]>https://xcp-ng.org/forum/post/46048https://xcp-ng.org/forum/post/46048Mon, 24 Jan 2022 03:02:28 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Mon, 24 Jan 2022 02:46:58 GMT]]>Hi,

Can anyone assist as to why my full backups are over 30 days old:
d7ff1037-c9d4-43d2-9aaa-3a69de3899ca-image.png

Here is my backup settings:
74ad1945-dfdb-4109-97f8-6db83aa3c475-image.png

I want to have a total of 7 backups per week (1 full plus 6 deltas).

]]>https://xcp-ng.org/forum/post/46046https://xcp-ng.org/forum/post/46046Mon, 24 Jan 2022 02:46:58 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Thu, 13 Jan 2022 14:50:43 GMT]]>@olivierlambert Thank you, I've changed the settings on b2 to "Keep all versions of the file (default)".

]]>https://xcp-ng.org/forum/post/45713https://xcp-ng.org/forum/post/45713Thu, 13 Jan 2022 14:50:43 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Thu, 13 Jan 2022 14:40:38 GMT]]>My gut feeling: avoid any interaction, because XO is not meant to have files removed by another program, this might destroy your backup.

]]>https://xcp-ng.org/forum/post/45712https://xcp-ng.org/forum/post/45712Thu, 13 Jan 2022 14:40:38 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Thu, 13 Jan 2022 14:39:08 GMT]]>@olivierlambert Thank you, didn't realize it's been that long. Just updated to latest commit (b78a9). I'll see in a week or so if the issue resolves itself.

I'm wondering if I should change the b2 Lifecycle settings from 7 days to only last version? Just not clear on how XO full & delta backups with the b2 Lifecycle settings interact with each other:

1d1a4a9a-c4fe-44fb-80e8-2af7f385028f-image.png

]]>https://xcp-ng.org/forum/post/45711https://xcp-ng.org/forum/post/45711Thu, 13 Jan 2022 14:39:08 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Thu, 13 Jan 2022 14:28:07 GMT]]>It's already 22 days old and more than 30 commits old. Please always check that your are on the latest master commit and rebuild before reporting an issue 🙂

]]>https://xcp-ng.org/forum/post/45706https://xcp-ng.org/forum/post/45706Thu, 13 Jan 2022 14:28:07 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Thu, 13 Jan 2022 14:25:28 GMT]]>@olivierlambert Good Morning, sorry didn't include that info:

XO from sources, commit 4e88c
xo-server 5.86.1
xo-web 5.91.0

]]>https://xcp-ng.org/forum/post/45705https://xcp-ng.org/forum/post/45705Thu, 13 Jan 2022 14:25:28 GMT<![CDATA[Reply to S3 Backup Retention Not Deleting Old Backups on Thu, 13 Jan 2022 08:25:28 GMT]]>Hi,

Do not interfere in file retention directly in Backblaze, that might remove legit files from XO backup.

Also, in order to help you, we must know if you are using XOA or XO from the sources, and if you are fully up to date 🙂

]]>https://xcp-ng.org/forum/post/45697https://xcp-ng.org/forum/post/45697Thu, 13 Jan 2022 08:25:28 GMT