Question: Delta backups and snapshot modes

aghering

I recently migrated my homelab from VMware workstation to XCP-ng.
My homelab runs various servers on Windows and Linux like AD, Exchange, Nextcloud, FreeNAS and more.

On setting up a delta backup schedule i can choose in the settings -> advanced for snapshot mode and i wonder what is best practice for server like mentioned above.

Is it save to create delta backups of a exchange, database and others server with snapshot mode on 'normal' while in 'running' or 'halted' state or is it best practice to backup with snapshot mode 'with memory' or 'offline'.

So far i know shutting down the VM and then run a backup is the safest way. But i want to avoid that because of the dependencies of the servers on each other (LDAP, storage repositories and NFS shares linked to NAS, Reverse Proxy, etc)

Running:
XCP-ng: 8.1.0
XO-server: 5.66.0
XO-web: 5.69.0
All VMs have management agent installed

tony

@aghering most VMs will be fine with normal mode while running, you don't need to halt them (XO may halt it for like a second or two anyway). Not all OSs support snapshot with memory, so even if you can do them when you roll them back you'll still need to restart the OS anyway meaning you lose data in the memory regardless. Offline is probably only needed if you can't get it to work in normal mode which should work almost 100% of the time.

aghering

Thank you @tony, based on your answer i now created delta backups with snapshot mode 'normal'. Tested the backup and restore with a couple of VMs Windows, Ubuntu and FreeBSB. With the windows and ubuntu VMs they where no issues. The FreeBSB with FreeNAS running throws an error: "UNIMPLEMENTED_IN_SM_BACKEND (OpaqueRef:9bfbc055-40c4-44a0-ab1a-9f3bac6dc718)".

Two things to note with the FreeBSB FreeNAS server:

There is second disk mounted, my data disk, to it. If possible can you exclude disks in a VM from a backup schedule?
There is no management tool installed on the VM. From my understanding freenas needs a different approach and before getting into that i want a backup.

tony

@aghering when does the error pop up, during backup or when you try to set it up?

There is no way to exclude disk from VM in the current version, you can make it skip the transfer phrase so no backup will be transferred using [NOBAK] feature but the snapshot will still happen.
I haven't never used FreeBSD so I can't say for sure but it seems like a simple installation for the guest tool. Can you try making a snapshot of the FreeBSD VM, just want to see which function causes the error.

aghering

Hi Tony,

Excuse me for my late reaction but today i looked into this issue again with no success

VM [NOBAK] option and snapshot error in XO
I added the [NOBAK] quotes to the name of the disk i want to skip the transfer for and fired a manual snapshot job. Almost instantly i get the same error as mentioned before, screenshot and error details below.

vm.snapshot
{
  "id": "b95f5a0e-3694-8066-ecb4-f72294d13f8a"
}
{
  "code": "UNIMPLEMENTED_IN_SM_BACKEND",
  "params": [
    "OpaqueRef:9bfbc055-40c4-44a0-ab1a-9f3bac6dc718"
  ],
  "task": {
    "uuid": "175c0c3f-d480-c769-31eb-423f8fe21aa9",
    "name_label": "Async.VM.snapshot",
    "name_description": "",
    "allowed_operations": [],
    "current_operations": {},
    "created": "20200923T14:40:20Z",
    "finished": "20200923T14:40:22Z",
    "status": "failure",
    "resident_on": "OpaqueRef:f03bc02e-9087-41f4-b0ac-c57f3f5220dd",
    "progress": 1,
    "type": "<none/>",
    "result": "",
    "error_info": [
      "UNIMPLEMENTED_IN_SM_BACKEND",
      "OpaqueRef:9bfbc055-40c4-44a0-ab1a-9f3bac6dc718"
    ],
    "other_config": {},
    "subtask_of": "OpaqueRef:NULL",
    "subtasks": [
      "OpaqueRef:c6ae9555-aec8-40ca-809a-4be54d852987"
    ],
    "backtrace": "(((process xapi)(filename ocaml/xapi/xapi_vm_clone.ml)(line 60))((process xapi)(filename list.ml)(line 106))((process xapi)(filename ocaml/xapi/xapi_vm_clone.ml)(line 94))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename ocaml/xapi/xapi_vm_clone.ml)(line 102))((process xapi)(filename ocaml/xapi/xapi_vm_clone.ml)(line 136))((process xapi)(filename ocaml/xapi/xapi_vm_clone.ml)(line 162))((process xapi)(filename ocaml/xapi/xapi_vm_clone.ml)(line 168))((process xapi)(filename list.ml)(line 117))((process xapi)(filename ocaml/xapi/xapi_vm_clone.ml)(line 170))((process xapi)(filename ocaml/xapi/xapi_vm_clone.ml)(line 381))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 35))((process xapi)(filename ocaml/xapi/xapi_vm_snapshot.ml)(line 31))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 100))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/xapi/rbac.ml)(line 236))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 83)))"
  },
  "message": "UNIMPLEMENTED_IN_SM_BACKEND(OpaqueRef:9bfbc055-40c4-44a0-ab1a-9f3bac6dc718)",
  "name": "XapiError",
  "stack": "XapiError: UNIMPLEMENTED_IN_SM_BACKEND(OpaqueRef:9bfbc055-40c4-44a0-ab1a-9f3bac6dc718)
    at Function.wrap (/opt/xo/xo-builds/xen-orchestra-202031082325/packages/xen-api/src/_XapiError.js:16:12)
    at _default (/opt/xo/xo-builds/xen-orchestra-202031082325/packages/xen-api/src/_getTaskResult.js:11:29)
    at Xapi._addRecordToCache (/opt/xo/xo-builds/xen-orchestra-202031082325/packages/xen-api/src/index.js:866:24)
    at forEach (/opt/xo/xo-builds/xen-orchestra-202031082325/packages/xen-api/src/index.js:887:14)
    at Array.forEach (<anonymous>)
    at Xapi._processEvents (/opt/xo/xo-builds/xen-orchestra-202031082325/packages/xen-api/src/index.js:877:12)
    at Xapi._watchEvents (/opt/xo/xo-builds/xen-orchestra-202031082325/packages/xen-api/src/index.js:1038:14)"
}

Installing guest tools on FreeBSB (FreeNAS)
The xcp-ng documentation page describes that you will need to install 2 ports manually on FreeBSB. On following the instructions for let say sysutils/xe-guest-utilities on freshports[dot]org page it says to run the following commands:

To install the port: cd /usr/ports/sysutils/xen-guest-tools/ && make install clean
To add the package: pkg install xen-guest-tools

First command seems to fail on 'make' and second command i entered just to see what the output would be.

olivierlambert

Ping @fohdeesha

fohdeesha

Hi, I have never attempted to install guest-utils on FreeNAS (FreeBSD based). Our documentation is for generic FreeBSD installs, however FreeNAS is highly customized and I believe they have most of the repositories disabled (which is why you get the repo not found error). It seems they don't have the required build tools to make either, but I could be wrong.

I'm not sure where you're seeing instructions to do "make install", we have a dedicated document section just for FreeNAS specifically that does not require running make. Try following it here: https://xcp-ng.org/docs/guests.html#freenas-truenas

tony

@aghering the [NOBAK] option only excludes the transfer process, it will still try to make a snapshot of the disks. There are many possible causes for your UNIMPLEMENTED IN SM BACKEND issue, the most likely causes are probably storage related. If it is not a critical VM, you can try to disconnect the second storage disk and try again. If that doesn't work you might have to forget the SR and try (you can reattach it again, just need to remember your sr-uuid.

Also, your /var/log/SMlog might reveal something.