-
Hi,
we are using S3 as a backup backend (delta backup around 6Tb). We verified that in a month we make about 40 million Copy Put List requests which cost almost as much as the storage.
Is this normal behavior or there is something wrong on our side?
Thanks -
6TiB is about 3M+ blocks of 2MiB. So having 40 millions requests means you had around 13 times changes on those blocks. For a month, it doesn't sound unreasonable
What's your S3 provider?
Also adding @florent in the loop.
-
Yeah this is relatively normal, it's using 2MiB blocks so there are going to be a lot of requests when you are doing this with large VMs or data.
Are these a ton of VMs or a few larger ones? If the later, might be worth moving that data to a NAS of some kinda, TrueNAS or whatever, and have the NAS handle the backup of it so that the VMs remain light and nimble.
Of course that only works IF you can do that.
-
O olivierlambert moved this topic from Xen Orchestra on
