Immutable S3 Backups (Backblaze) and Merging; A Little Confused

planedrop

Been doing some testing recently and hoping to get some feedback or a better understanding of how this works.

I am using Backblaze (S3 compatible) for some backup testing, I have the bucket setup with immutable storage (Object Lock as they call it) with a 30 day retention period for testing.

I created a backup job to test with this remote with the below settings:

2 VMs set to backup
Retention is 30
Full Backup Interval is 15

What I am confused about is, what happens when a merge needs to happen? If it's a locked object, you can't merge data into the full backup.

So once the 30 backup retention period is hit, how does the delta backup merge blocks into the full backup of the chain? That should be impossible if the object is write restricted.

However, I am not getting any errors when running this backup job beyond the 30 backup retention setting. Shouldn't it error out since it can't write to the full backup VHD?

Or there is maybe something I'm not understanding here?

I'm also wondering how this is managed long term, since the objects can't be deleted, XOA will (I presume) try to delete them, that will fail, and then they are just there in the bucket forever since XOA isn't aware of the immutable retention period to go cleanup later.

olivierlambert

Pinging @florent

planedrop

@olivierlambert @florent Just following up on this again, I am going to be doing some more testing this week to see if I can discover any issues. Still very curious about how this should be handled though.

florent

@planedrop
with your setup, you'll have 15 real days of immutability at worst.
here is a little schema with Key backup ( full) and delta . A chain of backup is a key backup and it's delta descendants
mutable backups are noted with a point

K.d.dKdddddKd // in this case the first chain is not protected, because parts of the chain are mutable, but the 2 most recent
K.KdddddKddd // 2 more backups here we have the longest immutables chains
K.dddddKdddd // here is the critical part with the shortest imutable chains : only the last one
K.ddddKdddd //the protetectd chain gros

to ensure a usable immutability of n day, you must have a full backup retention of at least n, and a backup retention of at least 2xn

I am not really sure on the error message that the UI show on backblaze, but the backup will be merged and deleted as soon as possible, when the object lock is lifted.
you can trust backblaze on their object lock, and you cna check the real number of backup stored by looking at the restore tab of XO