I found a weird thing, I'm trying to find the cause.

@shorian Ah sorry, I think S3 is completely broken at the moment.

@shorian Thank you.

I will look into the memory issue, that seems to be the clue into a hidden issue, thank you.

thanks, I've been inching along with external help, I just managed to compile the package a few minutes ago.
I will try to report here with an exact script to compile the RPM from nothing.

I am extremely sorry, but it seems to works on my machine

BUT, the error I see in your logs also exists at another customer site, and they also trigger it with a publicly available OVA.
If you can and want, would you mind trying to import 'watchguard-dimension_2_2.ova' https://software.watchguard.com/SoftwareDownloads?current=true&familyId=a2RF00000009On4MAE
and check if you get the same error ? you don't have to start the VM, we are just looking for vhd-tool output: vhd-tool: Non-zero size required in xensource.log

We will confer internally. in the mean time.

Nicolas.

yes, I'll have a look

Hi, sorry for the delay, I reproduce the error with chr-6.48.4.ova, I'm now starting to diagnose it.

@shorian thanks. If you could just try no concurrency as a proof that the code work, then your normal concurrency, to check if the issue is fixed that would be great.

This code is meant to be more vulnerable to losing the connection to S3, but I think it's a low risk. It's now using straight HTTP POST.

@shorian I would like to abuse your patience again, by asking you to test this branch: https://github.com/vatesfr/xen-orchestra/tree/nr-s3-fix-big-backups2

The concept is that the backup upload will happen without any sort of smart upload system or queue.

Thank you, Nico

I just had a getaddrinfo EAI_AGAIN error after 3h when trying a big backup towards AWS S3. According to stack overflow, it's generally a timeout on a DNS query. That seems close to your timeout error on the server side. I guess the immediate hypothesis is that the network side of XOA VM is choking on things.

In my last change I added support for keepalive in the AWS library, I will look in this direction.

@olivierlambert actually I saw the messages a few minutes before being tagged, and I was looking at the code by the time I got tagged.

@shorian Thank you very much, I'll dig deeper.

@deoccultist Thanks for the report, the fix is done, you can follow its route to a release here: https://github.com/vatesfr/xen-orchestra/pull/5574

Nicolas

nraynaud opened this pull request in vatesfr/xen-orchestra

closed fix(import/disk): allow uppercase extensions #5574

@shorian thank you I will look at all that.

Hello, the resolution should happen in this pull request: https://github.com/vatesfr/xen-orchestra/pull/5373

Nicolas.

nraynaud opened this pull request in vatesfr/xen-orchestra

closed feat(fs/backup) delete fs.createOutputStream() Allows S3 to back up metadata. #5373

@dgmcdona I see that the system did not recognize the disk, I will look into it deeper.

@cbritton privately sent me a file that exhibit the error, many thanks. I am searching for the bug now.

@tofaz thanks, I'll have a look at it.

you can follow the fix is in this pull request: https://github.com/vatesfr/xen-orchestra/pull/5085

nraynaud opened this pull request in vatesfr/xen-orchestra

closed feat(import/ova) allow import of gzipped vmdk disks. #5085

I have a local reproduction of the bug.

At first glance it looks like a stupid bug linked to the fact that there is more than one disk in the file.