Mystery solved:
There are numerous hosts under one master in the pool. I ran the yum update command on the master first, then rebooted, I then ran yum update on the rest and rebooted all but one. I could not reboot the last one until this morning (Saturday).

I rebooted the last one a few minutes ago and all is well.

Because I think its interesting, I'd love someone smart to explain the process of why other hosts could not do a snapshot because a different host in the pool had not been rebooted.

@Danp
WDE

@MajorP93
My bad....because of the word "rebooting" I took the statement "Were the hosts rebooting after the patches were applied?' to mean 'Were the hosts rebooting while the patches were being applied?'

To recap:
I did reboot after the patches had been applied.

Sorry !!

@MajorP93
I always reboot at the conclusion of the update and did as well this time.

@Danp
Nope!

Yesterday I upgraded to the most recent version of XCP-NG 8.3 via yum update.

Last night's backup is full of this error using the following:
Xen Orchestra, commit 8448b
Xen Orchestra, commit 1de60

I looked at the logs and they pointed to a snapshot error.

For the heck of it , I tried running a snapshot on any of the VM's. Each one fails with this error:

vm.snapshot
{
  "id": "0b65cf83-8309-3dcc-fa94-a511c77c96bd"
}
{
  "code": "NOT_SUPPORTED_DURING_UPGRADE",
  "params": [],
  "task": {
    "uuid": "71b630bb-d299-bab2-c2e9-065cd4bfef5c",
    "name_label": "Async.VM.snapshot",
    "name_description": "",
    "allowed_operations": [],
    "current_operations": {},
    "created": "20251219T14:33:39Z",
    "finished": "20251219T14:33:39Z",
    "status": "failure",
    "resident_on": "OpaqueRef:3ea53cca-30d6-7c19-3f77-3f7eb26c6675",
    "progress": 1,
    "type": "<none/>",
    "result": "",
    "error_info": [
      "NOT_SUPPORTED_DURING_UPGRADE"
    ],
    "other_config": {},
    "subtask_of": "OpaqueRef:NULL",
    "subtasks": [],
    "backtrace": "(((process xapi)(filename ocaml/xapi/xapi_vm_lifecycle.ml)(line 755))((process xapi)(filename ocaml/xapi/xapi_vm_helpers.ml)(line 1653))((process xapi)(filename fun.ml)(line 33))((process xapi)(filename fun.ml)(line 38))((process xapi)(filename ocaml/xapi/helpers.ml)(line 1788))((process xapi)(filename ocaml/xapi/xapi_vm_helpers.ml)(line 1652))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 1740))((process xapi)(filename ocaml/xapi/rbac.ml)(line 229))((process xapi)(filename ocaml/xapi/rbac.ml)(line 239))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))"
  },
  "message": "NOT_SUPPORTED_DURING_UPGRADE()",
  "name": "XapiError",
  "stack": "XapiError: NOT_SUPPORTED_DURING_UPGRADE()
    at Function.wrap (file:///opt/xo/xo-builds/xen-orchestra-202512190914/packages/xen-api/_XapiError.mjs:16:12)
    at default (file:///opt/xo/xo-builds/xen-orchestra-202512190914/packages/xen-api/_getTaskResult.mjs:13:29)
    at Xapi._addRecordToCache (file:///opt/xo/xo-builds/xen-orchestra-202512190914/packages/xen-api/index.mjs:1078:24)
    at file:///opt/xo/xo-builds/xen-orchestra-202512190914/packages/xen-api/index.mjs:1112:14
    at Array.forEach (<anonymous>)
    at Xapi._processEvents (file:///opt/xo/xo-builds/xen-orchestra-202512190914/packages/xen-api/index.mjs:1102:12)
    at Xapi._watchEvents (file:///opt/xo/xo-builds/xen-orchestra-202512190914/packages/xen-api/index.mjs:1275:14)"

I've tried restarting the toolstack on the pool master as well as the hosts.

Any ideas?

@Henrik
Thanks for the advice! I just did one.....lots of rebooting required.

After you run xclean, it then reboots. When it comes back, there is no mouse so, unless you can do everything from keystrokes, you have to reboot. When it comes back, you can then install the new drivers. It then reboots and now you can reset your original ip address (if it was a static IP address).

Whats the difference between this one and the the one created by "managementagentx64-9.4.2.msi" downloaded from "http://downloadns.citrix.com.edgesuite.net/17528/managementagent-9.4.2-x64.msi" or https://www.xenserver.com/downloads#lightbox-20160?

As of right now, using "managementagentx64-9.4.2.msi" I get "Management agent 9.4.2-178 detected" with the latest Citrix/xenserver.com drivers.

Is it better to start using this signed build of the XCP-ng Windows PV drivers?

Thanks!

@flakpyro
I'm good now!

I'm on build d8370 and last night all my backups ran without issue.

All is well for me ...thanks for the hard work!

@olivierlambert Thanks!

@tjkreidl Thank you!!!

@olivierlambert
Trying to avoid doing the shared NFS/Iscsi thing.

FWIW: The XO Community edition backups don't work as of build 6b263. If you roll back to commit 19412 they work fine.

The longer error is:
"Error: The last argument to .catch() must be a function, got [object Object]"

" "result": {
"message": "The last argument to .catch() must be a function, got [object Object]",
"name": "TypeError",
"stack": "TypeError: The last argument to .catch() must be a function, got [object Object]\n at Promise.caught.Promise.catch (/opt/xo/xo-builds/xen-orchestra-202506201840/node_modules/bluebird/js/release/promise.js:130:19)\n at /opt/xo/xo-builds/xen-orchestra-202506201840/@xen-orchestra/fs/src/abstract.js:717:43\n at next (/opt/xo/xo-builds/xen-orchestra-202506201840/@vates/async-each/index.js:90:37)"
}"

Is it possible for a script on one host to check if a VM is running on another host?

Id like to have a cron task that runs every few minutes on host A to see if a VM is running on host B. If its not, I want it to start a VM on A, if it is I want it to either carry on or to shut down the VM on host A.

Trying to get around running Opensense and don't have multiple WAN IP addresses for HA.

@olivierlambert
I’ve got a few VM’s that lock up from time to time. There is no rhyme nor reason (different hosts, different storage, etc).

As a result, I just have a job that restarts them at 5:30 am every day. I’d like to have the software tool software restart them but, in the occasion wher the VM is locked up tight as Fort Knox, I’d like the hard reset to happen.

Make sense ?

@pdonias Tis will sound weird but is there any way to have it try going through the guest tools first and, if it doesn't work, then do the hard-reboot of the VM?

@pdonias Thanks!

@pdonias
One more...what does "timeout" do?
Thank you!