@florent Thank you.
I just like things to be 'tidy' that's why I asked.

Is it OK for me to schedule a find and remove for *.keeper_* on the target share if they're over 8 hours old? Or do they need to stay?

@DustinB

As to why you're seeing "VM not found!" I don't know if that is a XO alert or something you added into the screenshot, I've never seen that so I can't speak to it.

I didn't add the text, all I did was highlight it with the red surround.

Just testing the backups again using the 'Delta' method and '[NOSNAP][NOBAK] Data drive' as the name of the second drive , feeling more hopeful about this one though.

I'm starting to like XO the more I use it, would be good to move to this instead of switching between 2 different XCP-ng Center EXEs. depending on the XCP-ng version on the host so I really hope I can get this to work.

The scheduled hourly snapshot while the guest is running looks a great feature, although I need to learn to walk with this first before I go that far

@DustinB Even though XO wasn't able to restore its own backup, fortunately XCP-ng Center was able to restore it from the .xva in the 'xo-vm-backups' folder that XO backup created:

The restored backup shows the drive name starts "[NOBAK]" so it wasn't excluded by XO when it first backed it up (presumably because it was a regular 'vm-export' type):

I'm guessing only subsequent incremental backups are going to exclude the "[NOBAK]" drive, i.e. the initial Full backup has to include drives marked "[NOBAK]"?

And do you have any idea why XO couldn't restore the backup it created the night before?
The only change was that I deleted the VM before trying to restore, but it can't be that the VM has to already exist on the server before XO can restore it surely?

Thanks

EDIT: Realised after watching Oliver's video, I should have used 'Delta' as the backup type >.<
Will try the experiments again

I'm clearly doing something wrong with XO though.

Left it backing up the VM about 2am this morning on my home test setup.
Said it worked OK.

So I deleted the VM and restored back to the same server... the restore says "Successful" but the VM isn't there. Haven't even got as far as trying to restore to a different server yet!

@DustinB I am currently experimenting with XO's backup - it's certainly comprehensive.
But unlike the current situation we've had for years, where there's a monthly ".xva" export which we can easily restore to any server (even at a different location) followed by a data restore, it is going to take some time to learn about the different features and different approach with XO.
I've created a backup job, and it exported to my local NAS without issue, even creating a ".xva" - so I intend restore that test to a different XCP-ng server to see how easy it is, then see how the incremental backups perform.

In the meantime, I tried my test script last night and it worked on my local setup perfectly (I had to add one more step to my logic above, xe snapshot-param-set is-a-template=false "uuid=$sViaSnapshotUuid")

So feeling confident I ran it on one of the 'live' servers in the early hours, but it failed because the server has the external "SCSI 4:0:0:0" drive which stops snapshots from working

I'll continue with the automated system already in place on the live setup, where each XenServer/XCP-ng does its own monthly export and each guest does its own encrypted backup as that's been working and rock solid for years (except for servers with external drives), but will invest spend time learning about the XO backup features you suggested as they look as though such an approach might work with these awkward drives that are currently stopping snapshotting, even with "[NOSNAP]" set.

Cheers!

@DustinB said in Is there an equivalent to [NOSNAP][NOBAK] for Exported VMs?:

I'm assuming these are iSCSI drives, and are on a SAN

The live ones are a either external USB drives passed thru to a specific guest (because they exceeded the 2TB limit currently on XCP-ng), or an internal NVME high capacity drive.

However, my test one is simply a virtual second disk on local storage.

I appreciate that '[NOBAK]' relates to backup only (which we don't currently use XO for, the data is backed up and encrypted as part of an overnight schedule on the guest itself).

I mistakenly thought that '[NOSNAP]' would hopefully exclude a drive from Exporting, but it didn't, which is fair enough.
In fact, it didn't exclude it from a snapshot either under XCP-ng 8.3 from the command line (xe vm-snapshot command), which surprised me as I thought this "[NOSNAP]" feature was implemented in the server itself, rather than clever bits added into XO.

I can't test it yet, but I've an idea that might work for our unusual situation without messing up the rest of the overnight too much, I think the 'xe vm-snapshot' command will create a logical snapshot (which includes references to all the local disks) instantly, without taking up much space at all, as from what I remember, the command was pretty much instant.
I can then get the script to run "xe vm-disk-list" just for the snapshot (rather than the guest VM) to obtain a list of all drives associated with it, and any that contain "[NOEXPORT]" it can issue a "xe vdi-forget" for that drive's UUID.

This should leave me with a logical snapshot that does NOT reference the drives where the names contain "[NOEXPORT]", so I can then export that logical snapshot to the target drive.

That's the theory anyway, and as it's only using 'xe' running locally on each server, it would probably work on the older XCP-ng / XenServer servers too, without having to get XO involved.

Hopefully I will get time to try it out when the servers aren't in use tomorrow at 1am.
It's just a theory though, so may end up back at square one

In our test lab, we have several different XCP-ng (and earlier XenServer 7.6) servers, each of which is allocated a different day of month to export at 1am via a simple bash script, using
xe vm-export vm=$sUuid filename="$sFnameTarget"

There are a couple of servers that we're currently having to do manually because they have 4TB drives full of data (backed up by other means) and we can't include them in the export.

What I'd like to do is to exclude these drives from the exports, so I've been experimenting on a 'Home environment' first, with just a single host running XCP-ng 8.3 and 2 guests: The first guest runs the Xen Orchestra Appliance (Xen Orchestra, commit 93a11) and the other guest is a simple Ubuntu guest with a second drive (see attachment).

On the 2 drive VM, I've set the drive name to "[NOSNAP][NOBAK] Second drive", as I've seen elsewhere on this forum, but running the 'xe vm-export...' from a shell still tried to include it.

I thought perhaps this "exclude drive" feature is only available from Orchestra rather than the host itself running 'xe' commands, so I tried exporting it from Orchestra via the Web UI - unfortunately, that didn't work either; it still tried to export the second drive.

I appreciate that '[NOBAK]' probably relates to backups rather than exports, but I thought the '[NOSNAP]' stood a good chance of working as (I thought) exports used the snapshot method first.

So my question is: Is there an equivalent '[NOEXPORT]' or similar to exclude the 2nd drive from exports by amending the drive name, similar to '[NOSNAP]'?

And if not, is there either a CLI xe command or Orchestra REST API equivalent of me manually detaching the second drive prior to the export then re-attaching it again please? As I don't want to have to do this in the early hours every month!

I appreciate that I'd have to get the bash script to retrieve and remember the ID for the drive to detach / re-attach after the export, but that's not a problem once I know the 'xe' or API commands to achieve the detach / attach.

Obviously I'd prefer the simpler '[NOSNAP]' type approach if there is an equivalent for exports under XO / the API?

Thank you.

@pdonias There was indeed a setting to force it to prompt the 'Save As'.
And it asked me where to save export..xva, I was able to select a network drive and click 'Save' and it did save to that location (I had to do it twice for some reason, bug in Edge I presume, but it ignored the request to save there the first time; I then repeated this with a different guest and it did the same).

The XO-lite's export method via the browser is slightly better than the full Xen Orchestra's export though; on the full XO, telling it to export which gives you a short life URL to copy and paste into a new tab, just then save to the target location feels very clunky.
And both web approaches aren't as logical as the simplicity of XCP-ng Center's 'Browse', select a target folder and that's it; pros and cons of native app vs browser UI limitations I guess.

Looking forward to seeing how all 3 products continue to evolve

@pdonias

I'm not sure to understand what you mean by "browser ran out of disk space". Do you mean your computer's disk?

Yes, local windows disk. Clicking on 'Export' started downloading into my browser's Downloads folder instead of asking where to save to - appreciate that might be a browser issue (Microsoft Edge v127.0.2651.105 64-bit)

Also, what do you mean by target? You'd like to be able to choose a destination folder on your computer?

I'd like to choose a UNC SMB/CIFs share on a NAS, which is where we export to using XCP-ng Center currently, e.g. \\192.168.123.123\VMs

@pctechsolution This would be very useful for us too.
In our testing environment, we have about a dozen XCP-ng 8.2.1 hosts, but also 6 XenServer 7.6 ones. The 20.04.01 version of XCP-ng Center happily talks to 8.2.1 and earlier, but the later versions of XCP-ng Center will only talk to 8.2.1, not the earlier XenServers.

Conversely, the 20.04.01 version of XenCenter won't talk to the new test 8.3 host, but the later one will.

So at the moment, we're having to jump between versions, which will become a pain as we migrate the hosts over to XCP-ng 8.3

The later one also seems to have lost the right-click->Paste on the console, which I didn't realise how often I used until it was gone

Have also tried XO-lite on the 8.3, as it's being pushed as a possible alternative to XenCenter when XO isn't available.
The XO-lite is getting better, but still a long way to go before it gives the functionality of XenCenter, many of the options don't do anything yet, and even the bits that do work aren't great - like not being able to specify the target of an export (browser ran out of disk space, unsurprisingly).

Hope Michael is able to get XenCenter to support the older XenServers as well as the XCP-ng, even if some functionality is missing - it will still be an ideal middle ground between XO-lite and the full blown XO (which we don't want to reduce resources from other guests to implement)