@florent Thank you for the update.
Posts made by frank-s
-
RE: s3 remote download speeds
@florent Hi, Have you been able to make any progress with this?
Thanks. -
RE: Custom email
Thanks for your replies. Since the advent of mirror backups I realised there was an opportunity to backup to several different USB disks where each disk is a different remote allowing rotation of the disks. The primary backup goes to additional internal storage which is then mirrored to the USB disks. Only one snapshot per vm is needed regardless of how many USB disks are in use. To do this I need to script the mounting process for these disks and send notification emails when a disk is mounted, unmounted (safe to unplug) or when no disk could be found.
This is my use case.
Thanks -
Custom email
Is there a simple way to send a custom email from the xo command line eg xo-cli?
-
RE: s3 remote download speeds
@florent Thank you so much for looking into this. The ability to saturate the download link will make a big difference - especially for on premises servers.
-
RE: s3 remote download speeds
@florent Thank you. I am a mere user - you devs are light years in front of me so I wouldn't know what to do with smapiv3...
I guessed I could probably download an entire folder from s3 using third party tools with parallel processing and then make a local remote from that to restore vms but this would be a 2 stage process. I can't help but feel that there is a more elegant solution. -
RE: s3 remote download speeds
@florent said in s3 remote download speeds:
We made the choice to optimize backup instead of restore, since there is more backups than restores.
I see. I understand that there would be more backups than restores - in an ideal world there would be no restores
Yet in the real world when a full restore is needed it is likely to be needed desperately. In this case I would probably prioritise restore over backup - especially where deltas are used as these would be smaller anyway... -
s3 remote download speeds
So I continue to test s3 remote. I have noticed that when I upload a backup it is able to completely saturate my upload bandwidth (40Mbps), however when I download (restore) I am lucky to get 20Mbps. I wondered if Wasabi was throttling my downloads so I installed Wasabi explorer on a windows box and tried downloading a large folder. Immediately it was much faster. Then I found the option for increasing the threads and I was able to saturate my download bandwidth (350Mbps). Can I ask how many threads the s3 remote uses when downloading and is there a way to tweak its performance?
Thanks. -
RE: S3 remote cannot set up with encryption
@florent I've been away for a few days so sorry for not replying sooner. Thank you all for your advice. I have still been unable to set up encryption on an s3 remote. I even tried a different provider (iDrive e2) but got the same error. I did manage to do it with backblaze b2 but the download speed for restore was too slow. I take onboard planedrop's point about https encrypting data in flight so if I proceed in production I will use the encryption tools provided by the s3 provider and ensure I always use https. Having said that I will probably wait till the s3 remote is no longer in beta.
Once again thanks everyone. -
RE: S3 remote cannot set up with encryption
Thank you. So what is the procedure? Do I have to initially back up to the unencrypted remote and then apply the encryption key?
-
RE: S3 remote cannot set up with encryption
This from the log created if I test the remote:
remote.test { "id": "74bf3202-064a-4b00-975b-34d9b7ad2904" } { "code": "ENOENT", "path": "/metadata.json", "message": "ENOENT: no such file '/metadata.json'", "name": "Error", "stack": "Error: ENOENT: no such file '/metadata.json' at S3Handler._createReadStream (/opt/xo/xo-builds/xen-orchestra-202402141057/@xen-orchestra/fs/src/s3.js:292:23)" }
-
RE: S3 remote cannot set up with encryption
I just noticed that my xo installation was 4 commits behind master so I ran the updater repeatedly till it was up to date. Unfortunately the same error persists.
-
RE: S3 remote cannot set up with encryption
@DustinB Thanks. I wouldn't like to send backup across the internet unencrypted and I would prefer not to trust Wasabi's encryption. The s3 remote I am setting up is for a mirror backup so one option I have tried is to encrypt the source (local) remote and then mirror that to Wasabi s3. That works although it seems to slow down local backup somewhat. I would prefer to encrypt the s3 mirror remote as the limiting factor there would be internet upload speed (40Mbps) so encryption wouldn't make any difference.
-
RE: S3 remote cannot set up with encryption
I can confirm that when setting up a local remote (SMB) using encryption everything appears to work OK. Just having a problem setting it up with Wasabi S3.
-
S3 remote cannot set up with encryption
Hi,
I am setting up an S3 remote with WASABI. If I don't use an encryption key it sets up fine. If I enter a 32 character hex key for encryption it fails to connect with the following message:
ENOENT: no such file '/metadata.json'
Am I doing something wrong?
Thanks -
RE: Compressed backups
@olivierlambert Thank you. Does this apply to the key backup as well as the increrments?
-
RE: Compressed backups
@DustinB
I couldn't see the Compression field under "delta backups" but now I see it for full backups. I guess then compression is not available for delta backup because it would screw up file merge when retention has been reached. Is this right? -
Compressed backups
Hi, Does delta (incremental) backup compress backups on the backup repository? Is there a way to enable this to reduce the amount of data to transfer?
Thanks. -
RE: Instant restore
@olivierlambert Thank you. Can I just say that you guys are right on the ball! I don't know of any other software forums where answers come back quite so instantly!
Many thanks.