Unable to export reliable backups

[bsodmike]

@olivierlambert The delta backup finally shows an error:

Duration: 18 minutes
Error: VDI_IO_ERROR(Device I/O errors)

[Danp]

@bsodmike Seems like you are still having issues with your FreeNAS setup. Do you have a different (non-FreeNAS) storage device that you can use for testing?

I noticed that your host0 is missing some patches. Probably best to address that, too. Have you tried doing a CR backup between the two hosts?

[bsodmike]

@danp Hi Dan, I am easily able to export to FreeNAS via xe vm-export. The only issue I'm facing right now is getting XOA to run backups to NFS (or even CIFS) shares.

Right now, I do not have any other hosts to backup to, but I fail to see the FreeNAS host I'm using as the failure point given above. Thoughts?

[bsodmike]

@danp Also FYI, XOA failing backups (to the NFS share mentioned above) should not create smaller files and go "green" marking the backup as successful.

These are all failed attempts on host0.

[Danp]

@bsodmike Are you using XOA (the pre-built appliance) or XO from sources?

[bsodmike]

@danp I tried both actually, but right now I've switched to install via source using https://github.com/ronivay/XenOrchestraInstallerUpdater

[bsodmike]

@danp Here's an example that I just ran of a manual export to a NFS share that I setup on the host,

[23:22 xcpng-host0-mgmt xcpng-manual-export]# xe vm-export vm=docker1 filename=/mnt/xcpng_data/xcpng-manual-export/docker1_`date +"%d%m%Y"`.xva
Export succeeded
[23:28 xcpng-host0-mgmt xcpng-manual-export]# pwd
/mnt/xcpng_data/xcpng-manual-export
[23:29 xcpng-host0-mgmt xcpng-manual-export]# ls -lah
total 8.7G
drwxr-xr-x 2 1001 1005   3 Dec 20 23:23 .
drwxrwx--- 9 1004 1005  12 Dec 20 23:16 ..
-rw------- 1 root 1005 36G Dec 20 23:28 docker1_20122020.xva
[23:29 xcpng-host0-mgmt xcpng-manual-export]# mount -t nfs
freenas-primary.intranet:/mnt/big-primary/xcpng_data on /mnt/xcpng_data type nfs (rw,relatime,vers=3,rsize=131072,wsize=131072,namlen=255,acdirmin=0,acdirmax=0,soft,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.0.0.149,mountvers=3,mountport=727,mountproto=tcp,local_lock=none,addr=10.0.0.149)

[olivierlambert]

Obviously there's something wrong in the setup. Basic backups or even delta issues like this would have trigger QA before a release, not talking about the 3000+ active users.

Now the question is: what's wrong exactly? Hard to tell, but I have suspicions on your storage itself. VDI_IO_ERROR(Device I/O errors) is related to XCP-ng issue on export/import.

[bsodmike]

@olivierlambert That makes sense. FYI I only got the VDI error when attempting delta backup via XO. xe vm-export has not provided any errors.

Just right now I was able to export a 35Gig XVA file, reimport using xe vm-import and the VM boots just fine.

All over the SAME NFS share that doesn't work with XOA. Hmm...

[olivierlambert]

Check dmesg in both host and XO VM to see if you can spot anything.

[bsodmike]

@olivierlambert I have a fedora workstation, which I could use to serve an NFS share. Hmm, I could try this to isolate the Freenas share?

[olivierlambert]

That would be interesting. This won't rule out everything but at least we might notice something different.