I'll try to answer to the questions

@JamesG the 32 chars are the key , you need to keep this key, if not you won't be able to restore anything. IF the key is wrong you'll have an error message when connecting to the remote.

I am glad to see that S3 is now stable enough for you all. To improve it again, we are planing to implement retry every times it is possible. It is already implemented during the reading of the backup in NBD, please give it a try.

There is also a test branch ( https://github.com/vatesfr/xen-orchestra/pull/6840 ) that should improve concurrency handling when using NBD reading and writing to S3
For now, you should test it on a separate job with a big VM

The merge is mostly a copy and delete of all the block (1-2MB each) that are not used anymore. This step could probably by speed up further, but we prioritized reliability here, since it can break the full backup chain it something go wrong ( and thanks @andrew for your time on this feature). You can set mergeBlockConcurrency to a higher value ( like 8 ) in the backups section of your config file

You can also increase writeBlockConcurrency in the backup section of the config file to speed up transfer, especially when coupled with the test branch

The easiest way to ensure your config file is not reset on each update is to create the config in ~/.config/xo-server/config.toml (use the home directory of the user running XO )
If you have a xoa, use /etc/xo-server/config.toml

[backups]
writeConcurrency = 32
mergeConcurrency = 8

fbeauchamp opened this pull request in vatesfr/xen-orchestra

draft feat(@xen-orchestra/backups): use parallel reading whith NBD + blocks to only one remote #6840

I think S3 can go out of beta

I am confident on the encryption protocol but I think we would need to have a little more polish on the ui before really graduating it to stable : for example the key length and format, how to handle encryption key change (you must not do it if there is any data already encrypted), backup/restore key. Maybe giving some choices on the algorithm used (I think some level of encryption level are illegals in some countries)

At first glance, it does not seem so easy.
First, we will implement this feature in XO Lite (soon). We will then see if there is a trivial way to adapt the changes in XO5

After a discussion with the XO team, the ability to edit from the UI: creation.user and date, has been added to the backlog.

@sluflyer06 the warning on XO is to discourage the mass. But you can really try

you can make a config.toml file in ~/.config/xo-server/config.*.toml , it should be loaded , and won't be changed by an XO update . https://github.com/JsCommunity/app-conf

@mauzilla There is no easy way to achieve that, it requires updating a number of other_config entries manually.

If you have a paid subscription, open a support ticket and I'll take care of it for you

@mauzilla Changing the name of the backup jobs or even the snapshots has no impact on backup execution.

@sluflyer06 For now , I get back to the first problem : NBD should be faster than xapi export ( or at least as fast)

now I 'm working on enabling insecure

edit : the not allowed is only to inform the user that it can't be set through XO, but it should be used for backup if present

@sluflyer06 can you test the speed with this branch based on master https://github.com/vatesfr/xen-orchestra/pull/6840

The backup job must have only one remote , which must be block based, and without a continuous replication

Warning : this is not production ready, this branch is only to find where the potential gains are.

fbeauchamp opened this pull request in vatesfr/xen-orchestra

draft feat(@xen-orchestra/backups): use parallel reading whith NBD + blocks to only one remote #6840

@sluflyer06 we did a rework on NBD, to be able to use it everywhere, paving the way for some nice features like retry on network error

but it may have an impact on block based backup. I will make a branch which should alleviate this today