S3 remote cannot set up with encryption

frank-s

Hi,
I am setting up an S3 remote with WASABI. If I don't use an encryption key it sets up fine. If I enter a 32 character hex key for encryption it fails to connect with the following message:
ENOENT: no such file '/metadata.json'
Am I doing something wrong?
Thanks

olivierlambert

Hmm question for @florent

frank-s

I can confirm that when setting up a local remote (SMB) using encryption everything appears to work OK. Just having a problem setting it up with Wasabi S3.

DustinB

I saw similar behavior on Xen Orchestra, commit 7cea4, but I just moved the encryption to my target, and cloud storage directly.

It's an Alpha feature so..

frank-s

@DustinB Thanks. I wouldn't like to send backup across the internet unencrypted and I would prefer not to trust Wasabi's encryption. The s3 remote I am setting up is for a mirror backup so one option I have tried is to encrypt the source (local) remote and then mirror that to Wasabi s3. That works although it seems to slow down local backup somewhat. I would prefer to encrypt the s3 mirror remote as the limiting factor there would be internet upload speed (40Mbps) so encryption wouldn't make any difference.

frank-s

I just noticed that my xo installation was 4 commits behind master so I ran the updater repeatedly till it was up to date. Unfortunately the same error persists.

olivierlambert

Do you have a more detailed error?

frank-s

This from the log created if I test the remote:

remote.test
{
  "id": "74bf3202-064a-4b00-975b-34d9b7ad2904"
}
{
  "code": "ENOENT",
  "path": "/metadata.json",
  "message": "ENOENT: no such file '/metadata.json'",
  "name": "Error",
  "stack": "Error: ENOENT: no such file '/metadata.json'
    at S3Handler._createReadStream (/opt/xo/xo-builds/xen-orchestra-202402141057/@xen-orchestra/fs/src/s3.js:292:23)"
}

florent

@frank-s
does this remote contains any data ? you can't change the encryption on non empty remotes
(even in this case, the error should be more useful)

frank-s

Thank you. So what is the procedure? Do I have to initially back up to the unencrypted remote and then apply the encryption key?

planedrop

@frank-s it's worth noting that you can setup S3 backup without encryption and you are not sending unencrypted data across the web, it's still covered by SSL/HTTPS, this feature is to encrypt the data before transmission so that it's encrypted when it gets to the target S3 bucket (meaning the S3 provider couldn't see anything you're storing). But the transmission of the data still uses encryption.

That is, as long as use HTTPS is enabled.

florent

@frank-s said in S3 remote cannot set up with encryption:

Thank you. So what is the procedure? Do I have to initially back up to the unencrypted remote and then apply the encryption key?

no, you should create an empty remote , add the key and start making backup

@planedrop you are right. What we propose here is "encryption at rest". We also use an authenticated algoriithm, it gives an additional bonus : a modified block will be detected on read, ensuring that if the backup is restored, it is exactly as it was at the source .

with the backup immutability of XO 5.91.2 ( for on premise backup repository) or object lock ( for s3 backup repository), this allow us to provide a 3-2-1-1-0 backup strategy, that is best eplained by veeam https://www.veeam.com/blog/321-backup-rule.html

frank-s

@florent I've been away for a few days so sorry for not replying sooner. Thank you all for your advice. I have still been unable to set up encryption on an s3 remote. I even tried a different provider (iDrive e2) but got the same error. I did manage to do it with backblaze b2 but the download speed for restore was too slow. I take onboard planedrop's point about https encrypting data in flight so if I proceed in production I will use the encryption tools provided by the s3 provider and ensure I always use https. Having said that I will probably wait till the s3 remote is no longer in beta.
Once again thanks everyone.