Compressed backups

frank-s

CJ

@olivierlambert Are you referring to the compression ratio when you mention performance? I turned it on for my remote and it tanked my transfer speeds compared to the standard sequential backup.

Since I'm using zfs for my remote, is there a way to turn off compression for the delta backups?

olivierlambert

Question for @florent

florent

@CJ said in Compressed backups:

@olivierlambert Are you referring to the compression ratio when you mention performance? I turned it on for my remote and it tanked my transfer speeds compared to the standard sequential backup.

Since I'm using zfs for my remote, is there a way to turn off compression for the delta backups?

you can turn compression off by setting vhd compression to none instead of brotly in your configuration file ,

[backups]
# Mode to use for newly created backup directories
vhdDirectoryCompression = 'brotli

writing vhd blocks is often slower than sequential, and compression should not have a lot of impact , but the merging is faster. So the best solution depends on the limiting factor on your infrastructure
(also this storage mode have been made for s3 which do not like modifying the files )

abudef

@olivierlambert said in Compressed backups:

Compression is enabled by default for incremental backups.

Hi, if I configure Delta Backup and run a backup for the first time, a full backup of the affected virtual machines is performed. This initial full backup is also compressed?

olivierlambert

IIRC, everything is compressed using Brotli by default, including the full.

abudef

If I understand correctly, for compression to work for Incremental (Delta) backup, remote storage must have the option "Store backup as multiple data blocks instead of a whole VHD file" enabled, otherwise vhd files are stored without compression. Is that right?

olivierlambert

I think we compress everything on flight, but I'm not sure, @florent could tell

abudef

At least the first backup, i.e. the full backup (and maybe every key full backup milestone later) does not look like it is compressed. If the "Store backup as multiple data blocks instead of a whole VHD file" option is off, a .vhd of 116 GiB is stored, if it is on, the whole data block structure is only 67 GiB.

abudef

In addition, I have verified that if the option "Store backup as multiple data blocks instead of a whole VHD file" is enabled, the restore from the data blocks structure takes about 2-3 longer than from the .vhd if this option is disabled (I tested the health check restore).

abudef

(The idea behind is to have both a compressed delta backup (the backup repository itself is backed up to another location via vpn, so the size of the backup matters) and the fastest possible restore option.)

frank-s

@abudef I just had a backup failure. This triggered a full (key) backup on the next run which transferred 165GiB. When I rdpd into the vm I could see that the vm contained 345GiB of data including the system drive. I conclude, therefore that compression is working for full key backups at a ration of around 2:1.
I hope this helps.

abudef

@frank-s Do you have the "data block" option enabled for remote or not?

frank-s

@abudef Yes, data blocks is enabled on the remote.

abudef

@frank-s This confirms my observations and tests that for compression to be active, data blocks must be enabled for remote.