XCP-ng 8.3 updates announcements and testing

mthird

Spoke to soon. While the updates succeeded, one of the nodes is rebooting every few minutes due to an HA self-fence.

manilx

This is a new one:

Updated all our 3 pools:
ScreenShot 2026-08-28 at 07.20.40.png
Backup via CLI, runs XOA

Then DR.
ScreenShot 2026-08-28 at 07.20.28.png
Tasks showed RPU on pool PRD instead of DR. RPU finished just fine but task stayed.

Then RPU on PRD. Task did not change. Finished also fine. task stayed.

gduperrey

@manilx If the pools are up to date, I would try restarting the XO VM to see if the task persists.

If it does persist, it might be best to open a separate topic for the XO team.

manilx

@gduperrey Restarted and the task persisted. Deleted it from the list.

acebmxer

1 out of 3 pools at work failed rolling pool update.

When I try to put host 2 into maintence mode to finish updates i get this error...

Support Ticket - Ticket#7763405

host.setMaintenanceMode
{
  "id": "60701efd-089c-4822-97c0-1a1057f3f9aa",
  "maintenance": true
}
{
  "code": "VM_REQUIRES_SR",
  "params": [
    "OpaqueRef:841e7606-d545-cb71-f67c-e48b5114a1e8",
    "OpaqueRef:0f356ee4-62f7-9608-8be5-df68d9a5cbb2"
  ],
  "task": {
    "uuid": "a24f5565-b2cb-f392-8095-1beceb7a4a17",
    "name_label": "Async.host.evacuate",
    "name_description": "",
    "allowed_operations": [],
    "current_operations": {},
    "created": "20260828T19:57:29Z",
    "finished": "20260828T19:57:29Z",
    "status": "failure",
    "resident_on": "OpaqueRef:43c121e7-4ba4-193b-1e41-08c0f7e15690",
    "progress": 1,
    "type": "<none/>",
    "result": "",
    "error_info": [
      "VM_REQUIRES_SR",
      "OpaqueRef:841e7606-d545-cb71-f67c-e48b5114a1e8",
      "OpaqueRef:0f356ee4-62f7-9608-8be5-df68d9a5cbb2"
    ],
    "other_config": {},
    "subtask_of": "OpaqueRef:NULL",
    "subtasks": [],
    "backtrace": "(((process xapi)(filename ocaml/xapi/xapi_host.ml)(line 629))((process xapi)(filename hashtbl.ml)(line 159))((process xapi)(filename hashtbl.ml)(line 165))((process xapi)(filename hashtbl.ml)(line 170))((process xapi)(filename ocaml/xapi/xapi_host.ml)(line 625))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/rbac.ml)(line 228))((process xapi)(filename ocaml/xapi/rbac.ml)(line 238))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))"
  },
  "message": "VM_REQUIRES_SR(OpaqueRef:841e7606-d545-cb71-f67c-e48b5114a1e8, OpaqueRef:0f356ee4-62f7-9608-8be5-df68d9a5cbb2)",
  "name": "XapiError",
  "stack": "XapiError: VM_REQUIRES_SR(OpaqueRef:841e7606-d545-cb71-f67c-e48b5114a1e8, OpaqueRef:0f356ee4-62f7-9608-8be5-df68d9a5cbb2)
    at Function.wrap (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/_XapiError.mjs:16:12)
    at default (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/_getTaskResult.mjs:13:29)
    at Xapi._addRecordToCache (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1229:24)
    at file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1263:14
    at Array.forEach (<anonymous>)
    at Xapi._processEvents (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1253:12)
    at Xapi._watchEvents (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1460:14)"
}

SR are not connected to master host after reboot from updates. When try to connect them i get error...

Screenshot 2026-08-28 160401.png

pbd.connect
{
  "id": "e96ec5d8-c70f-c877-4263-f2cf763e7048"
}
{
  "code": "SR_BACKEND_FAILURE_108",
  "params": [
    "",
    "Unable to detect an NFS service on this target.",
    ""
  ],
  "task": {
    "uuid": "5574bfe5-1874-35f8-f53b-c9fc1abaebb5",
    "name_label": "Async.PBD.plug",
    "name_description": "",
    "allowed_operations": [],
    "current_operations": {},
    "created": "20260828T20:03:19Z",
    "finished": "20260828T20:03:22Z",
    "status": "failure",
    "resident_on": "OpaqueRef:43c121e7-4ba4-193b-1e41-08c0f7e15690",
    "progress": 1,
    "type": "<none/>",
    "result": "",
    "error_info": [
      "SR_BACKEND_FAILURE_108",
      "",
      "Unable to detect an NFS service on this target.",
      ""
    ],
    "other_config": {},
    "subtask_of": "OpaqueRef:NULL",
    "subtasks": [],
    "backtrace": "(((process xapi)(filename ocaml/xapi-idl/storage/storage_interface.ml)(line 455))((process xapi)(filename src/lib/idl.ml)(line 558))((process xapi)(filename ocaml/xapi/storage_utils.ml)(line 148))((process xapi)(filename lib/backtrace.ml)(line 251))((process xapi)(filename ocaml/xapi/xapi_pbd.ml)(line 196))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 141))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 6061))((process xapi)(filename ocaml/xapi/rbac.ml)(line 228))((process xapi)(filename ocaml/xapi/rbac.ml)(line 238))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))"
  },
  "message": "SR_BACKEND_FAILURE_108(, Unable to detect an NFS service on this target., )",
  "name": "XapiError",
  "stack": "XapiError: SR_BACKEND_FAILURE_108(, Unable to detect an NFS service on this target., )
    at Function.wrap (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/_XapiError.mjs:16:12)
    at default (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/_getTaskResult.mjs:13:29)
    at Xapi._addRecordToCache (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1229:24)
    at file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1263:14
    at Array.forEach (<anonymous>)
    at Xapi._processEvents (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1253:12)
    at Xapi._watchEvents (file:///usr/local/lib/node_modules/xo-server/node_modules/xen-api/index.mjs:1460:14)"
}

Update - bad port on switch.......

Update 2 - After updating synology to latest version i am no longer able to mount nfs 4.1 shares. I can not create one using 4.1. I will let me create with 4.0 not 4.1 How to remount NFS that were mounted as 4.1 now as 4.0?

ALl VMS down at this location currently.

Update 3 - Resolved... Synology update broke NFS 4.1. I had to ssh into synology and edit /etc/nfs/syno_nfs_conf and add nfs_minor_ver_enable=1 Now SR's mounted.

Update 4 - Found Patch notes about NFS version change. -
Version: 7.0.1-42218
NFSv4.1 and related advanced functions (multipathing) are no longer supported on Synology NAS models with the following package architectures: Alpine, Alpine4k, armada38x, Avoton, Braswell, Bromolow, Cedarview, Grantley, Monaco. If you had previously enabled NFSv4 and NFSv4.1, the maximum NFS protocol is set to NFSv4 by default after the update. For more information about the package architecture of your model, please refer to this article.

Our unit - RS1219+ Intel Atom C2538 Avoton

XCP-ng-JustGreat

I deferred applying the preview patches this time in order to try my luck again with RPU. Unfortunately, it did not work for me again as it has not in the past. My results are similar to others here: primary host patch application went fine including the reboot. However, once it began doing a secondary host, it started throwing errors e.g. CANNOT_EVACUATE_HOST and VM_REQUIRES_SR etc. However, manually putting the host in maintenance mode from the GUI evacuated each host just fine and I was able to apply the patches and reboot each subsequent host from the XO GUI. Also, as with others here, the RPU task hung in the task list and even a reboot of the XO VM would not clear it forcing me to delete the task using xo-cli e.g. xo-cli rest del tasks.
ENVIRONMENT: Home lab consisting of 4 x Dell OptiPlex 7040 i7-6700 SFF hosts, 48GB RAM each, 10 Gbps storage connections to a TrueNAS home-built NAS via NFS and XO from source (XOS) using @ronivay build script on AlmaLinux 10.2 minimal install VM with XO commit 6a441 compiled on 2026-08-28 from master branch. FWIW, RPU functionality remains unavailable to me, though obviously, this is not a showstopper for a 4 x host home lab pool. If there is anything I can do to help resolve this, please let me know as I remain an enthusiastic proponent of the Vates virtualization stack.

gduperrey

Hello @acebmxer,

I can see from the support ticket and the summary that the issue appears to be resolved and was linked to the NFS update on the Synology. It’s great that you were able to find the solution.

gduperrey

Hello @XCP-ng-JustGreat,

I cannot comment on RPU issues via XO. I would recommend creating a new topic to report the problem directly to the XO team so they can analyze it.

As you can see, we recommend validating updates via the command line directly on XCP-ng before releasing them. That is a different scenario.

However, if several of you are experiencing RPU issues with XO, a dedicated topic will allow that team to investigate the situation and consult other teams if necessary. This would ensure the issue is better addressed and analyzed

manilx

@gduperrey https://xcp-ng.org/forum/topic/12439/rpu-issue

stormi

@mthird said:

Spoke to soon. While the updates succeeded, one of the nodes is rebooting every few minutes due to an HA self-fence.

Could you open a dedicated thread and ping me there?