@olivierlambert I've dug into the issue a bit more, and it just gets stranger. Downloading to my Macbook on the same network gets ~4MB/s. This means there is something strange going on with the control domain around remote downloads. When I test the disk speed (which is an Samsung NVMe), the speed is 1.7GB/s: [14:31 xcp-host ~]# dd if=/dev/zero of=testfile oflag=direct bs=1M count=5000 status=progress 4998561792 bytes (5.0 GB) copied, 3.000607 s, 1.7 GB/s 5000+0 records in 5000+0 records out 5242880000 bytes (5.2 GB) copied, 3.15049 s, 1.7 GB/s When I test the network speed with Ookla's speedtest-cli it confirms a 1G connection, and a 1G transit to Europe (checked this twice because 1G Atlantic transits are rare): [14:44 xcp-host ~]# speedtest -s 2023 Speedtest by Ookla Server: LaFibre.info - Lyon (id = 2023) ISP: Bell Canada Latency: 108.05 ms (0.49 ms jitter) Download: 960.47 Mbps (data used: 1.5 GB ) Upload: 172.11 Mbps (data used: 293.4 MB ) Packet Loss: 0.0% Result URL: https://www.speedtest.net/result/c/b8f1da9f-5c57-4ad8-ad3b-729e1625affe On my Macbook (on 802.11ac) I can download an Ubuntu image at ~14MB/s ➜ wget http://ubuntu.mirrors.ovh.net/ubuntu-releases/21.10/ubuntu-21.10-desktop-amd64.iso --2021-11-16 14:49:41-- http://ubuntu.mirrors.ovh.net/ubuntu-releases/21.10/ubuntu-21.10-desktop-amd64.iso Resolving ubuntu.mirrors.ovh.net (ubuntu.mirrors.ovh.net)... 213.32.5.7 Connecting to ubuntu.mirrors.ovh.net (ubuntu.mirrors.ovh.net)|213.32.5.7|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 3116482560 (2.9G) [application/octet-stream] Saving to: ‘ubuntu-21.10-desktop-amd64.iso’ ubuntu-21.10-desktop-amd64.iso 3%[=> ] 107.27M 14.4MB/s eta 4m 14s However on the control domain the download speed is considerably lower: [14:52 xcp-host ~]# wget http://ubuntu.mirrors.ovh.net/ubuntu-releases/21.10/ubuntu-21.10-desktop-amd64.iso --2021-11-16 14:53:03-- http://ubuntu.mirrors.ovh.net/ubuntu-releases/21.10/ubuntu-21.10-desktop-amd64.iso Resolving ubuntu.mirrors.ovh.net (ubuntu.mirrors.ovh.net)... 213.32.5.7 Connecting to ubuntu.mirrors.ovh.net (ubuntu.mirrors.ovh.net)|213.32.5.7|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 3116482560 (2.9G) [application/octet-stream] Saving to: ‘ubuntu-21.10-desktop-amd64.iso.1’ 3% [==> ] 106,910,815 3.20MB/s eta 16m 47s Have you ever see something like this before? I saw you had some discussion on another thread regarding slow writes in a VM, but this looks like it affects the control domain as well: https://xcp-ng.org/forum/topic/4865/slow-disk-writes-inside-vm-on-8-2

Well, it's a bit hard to tell you directly what to do vs XOA, because on XOA you are already on a validated OS (Debian) with some packages installed. On which OS did you install Xen Orchestra in the first place?

Sorry for the huge delay, I just stumbled on this thread It's indeed the case now, XO automatically check the integrity and clean any unused files in VM directory on a remote when that VM is backed up.

Hi, Try to disable/enable back the auto power on for one VM, and try again.

@olivierlambert said in Commands in Xen Orchestra Jobs no longer working: The best solution is to rely on XOA, you know I agree with @olivierlambert! Personally, I'm using XO from sources in my home lab environment -- nothing is "production" and I'm mostly having fun trying to give back to the open source community.

@Danp - thank you. Yes, I did, but that was not helping in this case. Found the solution: Both machines are on a hostserver, which has NOT all networks. So the real reason must be a network problem, therefore a quite missleading error message After moving both VMs to the master of this pool (which is connected to the same network where the xoa resides), everything went fine. Same storage, same vhds.

@indyj The file restore issue was due to installing XO on FreeBSD (fine for VM restores, not file extraction). I then installed XO on Ubuntu 20, and this worked for restoring files from Linux VMs but not Windows VMs. Then I installed on CentOS 8 and had the same problem as Ubuntu. I then installed on Fedora 34 and everthing works fine.

Just hold on for this month release

Yes, it should be safe enough to try importing the backup, then starting the VM without network (ie VIFs) and see if it stays up for a few minutes. I think we could start by adding a manual feature and then see in a second step how we could automate it.

Also, the user will be created only on first connection. Maybe you are thinking about the plugin in a way that's not how it behave.

Sure, please keep us posted on your experiments If it's a bug, we'll have more info to be able to work on it!

@olivierlambert many thanks for the very quick response, it is indeed XO from sources. I think the burst explanation makes sense. Also, thanks for you and your teams continued development and support with both XCP-NG and XO.

Source: https://github.com/vatesfr/xen-orchestra/issues/5969 olivierlambert created this issue in vatesfr/xen-orchestra closed Save all associated host IPs to a pool and use them as fallback for XAPI connection #5969

And the IRC channel is #xcp-ng on oftc.net if you prefer IRC.

Hi, thank you for your support. I was out of Office for two weeks and will test this soon. I will give you feedback. Regards, Torsten

Thank you for your help this is the solution i was searching for, I appriciate this much.

What do you mean "how it works"? It's the same thing than before, just "isolated" from the rest, working in a... worker. And yes, the long run objective is the ability to run it as a service on the remote directly, reducing a lot the merge time (because latency is the worst enemy of merging).

@olivierlambert said in Backup does not work if one host in the pool is offline: When you connect to a working, machine, yes it does But XAPI connection isn't persistent: it means, when the pool is disconnected, XO has no recollection whatsoever on who was the "real" master the last time it was connected. The only info we store is the IP address you entered in Settings/server. I agree, functionally speaking, it can be limiting in your case. A viable option would be to save all existing hosts IPs in the XO database, as fallback if we can't connect to the IP you entered. This is not a very common case, that's why it wasn't done before Looks like a good suggestion. Thanks. In my case I migrated everything from the poolmaster to the second host, then changed the second host to pool master, waited an hour and then shut down the old pool master (now just a pool member). XOA was online during the entire process and I had forgotten to check the server list.

That's because the effort to change this will be taken into account in XO 6 Alternatively, you can indeed contribute, this will be very welcome!

Hi ! We are experiencing these kind of problems because we activated Continous Replication and frequency was way too low. Now I want to eliminate all those files in the chain, can I do it manually from XCP-NG Center or XOA ? Thanks !!! [image: 1635327169557-c2aa1fb3-2ed2-41cf-b242-e2a23e6832e4-image.png]