xoa slow export
-
It's hard to tell slow or not without knowing all the context
Using GZIP compression, it's "normal". Then you have network speed between the host and XO, then between XO and the web browser."Slow" alone doesn't mean anything!
-
@akurzawa said in xoa slow export:
It seems that this is impossible for xoa to export 2 o more vm at once
Indeed, for the time being, XO limits to 2 concurrent exports on any given pool, this will change in the near future.
-
@akurzawa said in xoa slow export:
I've endless export task
Should not cause any issues and it will automatically be destroyed by the XenAPI after 24 hours.
-
@julien-f said in xoa slow export:
@akurzawa said in xoa slow export:
I've endless export task
Should not cause any issues and it will automatically be destroyed by the XenAPI after 24 hours.
wow just 24 hours? Do you know that I cannot shotdown the host right now nor my UPS in case of power failure?
-
@akurzawa I'm not sure but restarting
xo-servermight do the trick as well. -
Toolstack restart will cancel the task.
-
Did you do any tests with XCP-ng Center? Just to get any numbers.
I have one VM on XS that needs to be exportet regulary, which hits about 800 GB in ~22 hours. It always starts reasonably quick and then becomes slower by time - but 800 vs. 8... -
@cg said in xoa slow export:
Did you do any tests with XCP-ng Center? Just to get any numbers.
I have one VM on XS that needs to be exportet regulary, which hits about 800 GB in ~22 hours. It always starts reasonably quick and then becomes slower by time - but 800 vs. 8...In 90% cases I'm obsrving the sam thing - export either in xoa or xcp-ng center is getting slowe over time... idk whats the reason of that.
-
As Citrix barely ever gave any insights... maybe @olivierlambert can bring light into that.
-
We aren't doing any test with XCP-ng Center because we don't use it at all
You can use xeto compare (xeCLI is basically another XAPI client -like XO or XCP-ng Center- which can run on the same host or even elsewhere) -
Regardin export from xoa I the highest speed that I've seen was 40MB/s on raid10 setup, xoa and vm on the same host. What could be the reason of that? Host is almost always idle, just very basic workload.
-
Export from XOA to where? Your browser, a remote?
-
Export from xoa, Gb LAN, latest Chrome, Windows
-
@olivierlambert It's basically the same. I almost only export via scripts/xe.
-
@akurzawa in this case, data is going from the SR to the host, then from the host to XOA, and finally from XOA to your PC with your browser. As you can see there is a lot of "hops" before being stored in your hard drive. And because it's a stream (there isn't any intermediate storage), it means it's going to the speed of the slower element (back pressure).
In general, Xen Orchestra isn't the bottleneck itself because it just "passes" the stream. The most common bottleneck is in general the host itself. Using VM export with
zstdcan yield good results.@cg so the bottleneck can be from your SR to your host, the host itself, or your host to the storage destination.
-
@olivierlambert: I doubt. It wouldn't explain why it's getting slower by time and while still using 100 % of once CPU core, the dataflow becomes constantly slower. It might be an IRQ/IO problem on hardware side, but that should be constant.
I monitored network and storage: Both is mostly idling.
It has 10/15k SAS RAID on both sides and 10G Ethernet and running only one export in the end. -
I said "the host itself", which could be a hardware issue. Obviously the bottleneck is somewhere
-
I agree with both of you, the problem is not on XO neither on hardware. I encountered this problem too, I tried many things (encryption off/on, no compression), same results. Like @cg said, the disks are idling also (99% idle)
I tried mostly tried on the hosts by itself, the templating system seems to work in the same way as export, and this was terribly slow for local storage in terms of throughput (25/30 Mbytes/s max on 15k SAS RAID 10). I put a bet on a XenServer core limitations/bottleneck (?), it could be some kind of a software limitation : https://discussions.citrix.com/topic/374896-exporting-vms-there-seems-to-be-a-speed-difference-between-xencenter-and-vm-export/
-
Depends on what are we talking about:
- yes, export speed is often "software limited" because how a XVA is made
- I'm not aware about an issue regarding speed being reduced progressively during the transfer
-
@olivierlambert maybe it's "generally" slow, but on the beginning the speed is higher due some buffering during file download
