XCP-ng 8.3 updates announcements and testing

[lmolina]

Thank you everyone for your tests and your feedback!

The updates are live now.

Note that the update includes an additional kernel security update. Another vulnerability was disclosed recently, we will provide provide the update soon. Checkout the blog post for detailed information: https://xcp-ng.org/blog/2026/11/03/november-2026-security-and-maintenance-updates-1-for-xcp-ng-8-3-lts/

[MajorP93]

Hello,

thanks for publishing these updates!

However did you guys change anything regarding rolling pool update?

I am using latest XO from sources.

It's the third time I am getting this:

I am using dynamic memory control.

During last few rounds of updates (last few months) rolling pool update was working just fine.
This is the third VM today for which RPU is complaining about ballooning / DMC.

For the first two VMs I disabled DMC and re-tried RPU.
It progressed to the next VM but then failed again with the error shown on the screenshot.

In the past RPU was working fine with DMC hence the question regarding RPU changes?

Thanks and best regards

[MajorP93]

Aaaaand it happened for the next VM.
Apart from updating XO I did not change anything...
It looks like RPU is failing on every single VM that has DMC enabled...
To me it looks like a regression

//EDIT:
the weird thing is that it fails with the same error even with DMC disabled...
It also fails when manually trying to live migrate the VM.
This particular VM is a Windows Server VM, XCP-ng tools version: Management agent 9.2.385-0
What leaves me baffled is that all of this has been working just fine for several months...

vm.migrate
{
  "vm": "25cf2537-c8f9-a3ef-ca68-6621351544c1",
  "migrationNetwork": "1ee81d68-6614-f134-f4b4-6bb5399bf66a",
  "targetHost": "d984660e-c9a9-4981-aac8-de3df67833a9"
}
{
  "code": "BALLOONING_TIMEOUT_BEFORE_MIGRATION",
  "params": [
    "OpaqueRef:0b74d118-6c0d-aed2-ca21-99f0f7d5c813"
  ],
  "task": {
    "uuid": "dd29dc7b-62a6-1820-79d7-f447b7a5dcf6",
    "name_label": "Async.VM.pool_migrate",
    "name_description": "",
    "allowed_operations": [],
    "current_operations": {},
    "created": "20261103T16:17:54Z",
    "finished": "20261103T16:20:44Z",
    "status": "failure",
    "resident_on": "OpaqueRef:73f0c67e-d88c-6ed0-9070-ca96f899319f",
    "progress": 1,
    "type": "<none/>",
    "result": "",
    "error_info": [
      "BALLOONING_TIMEOUT_BEFORE_MIGRATION",
      "OpaqueRef:0b74d118-6c0d-aed2-ca21-99f0f7d5c813"
    ],
    "other_config": {},
    "subtask_of": "OpaqueRef:NULL",
    "subtasks": [],
    "backtrace": "(((process xenopsd-xc)(filename ocaml/xenopsd/xc/xenops_server_xen.ml)(line 2802))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 103))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 111))((process xenopsd-xc)(filename ocaml/xenopsd/xc/xenops_server_xen.ml)(line 2859))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 103))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 111))((process xenopsd-xc)(filename ocaml/xenopsd/xc/domain.ml)(line 2122))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 103))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 111))((process xenopsd-xc)(filename ocaml/xenopsd/xc/domain.ml)(line 2121))((process xenopsd-xc)(filename ocaml/xenopsd/xc/emu_manager.ml)(line 239))((process xenopsd-xc)(filename ocaml/xenopsd/xc/emu_manager.ml)(line 244))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xenopsd-xc)(filename ocaml/xenopsd/xc/domain.ml)(line 2257))((process xenopsd-xc)(filename result.ml)(line 23))((process xenopsd-xc)(filename ocaml/xenopsd/xc/domain.ml)(line 2250))((process xenopsd-xc)(filename ocaml/xenopsd/xc/xenops_server_xen.ml)(line 2853))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 103))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 111))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 103))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 111))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2589))((process xenopsd-xc)(filename list.ml)(line 121))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2582))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2973))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xenopsd-xc)(filename ocaml/libs/stunnel/stunnel.ml)(line 439))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2986))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xenopsd-xc)(filename ocaml/libs/stunnel/stunnel.ml)(line 439))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2885))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 103))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 111))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3432))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3442))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3463))((process xenopsd-xc)(filename ocaml/xapi-idl/lib/task_server.ml)(line 192))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3483))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3654))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 355))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 361))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 386))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 527))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3492))((process xapi)(filename lib/backtrace.ml)(line 110))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 551))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 141))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 2605))((process xapi)(filename ocaml/xapi/rbac.ml)(line 228))((process xapi)(filename ocaml/xapi/rbac.ml)(line 238))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))"
  },
  "message": "BALLOONING_TIMEOUT_BEFORE_MIGRATION(OpaqueRef:0b74d118-6c0d-aed2-ca21-99f0f7d5c813)",
  "name": "XapiError",
  "stack": "XapiError: BALLOONING_TIMEOUT_BEFORE_MIGRATION(OpaqueRef:0b74d118-6c0d-aed2-ca21-99f0f7d5c813)
    at Function.wrap (file:///etc/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
    at default (file:///etc/xen-orchestra/packages/xen-api/_getTaskResult.mjs:13:29)
    at Xapi._addRecordToCache (file:///etc/xen-orchestra/packages/xen-api/index.mjs:1374:24)
    at file:///etc/xen-orchestra/packages/xen-api/index.mjs:1408:14
    at Array.forEach (<anonymous>)
    at Xapi._processEvents (file:///etc/xen-orchestra/packages/xen-api/index.mjs:1398:12)
    at Xapi._watchEvents (file:///etc/xen-orchestra/packages/xen-api/index.mjs:1605:14)"
}

[acebmxer]

@lmolina

I applied updates and nothing broken for me to report. I have done delta backup and a full backup both were successful.

[JeffBerntsen]

Updates applied to my systems, test and live. All is working well so far.

[Andrew]

@lmolina All updated. RPU successful. Systems running normally.

[gduperrey]

@MajorP93

We discussed your feedback yesterday, but we don't have an explanation at the moment. Our Windows expert tried to reproduce the behavior but was unable to trigger this error.

Furthermore, we haven't encountered any issues on our end with the RPU (Rolling Pool Update) operations performed during our tests, whether done manually or via XO. It is possible, however, that we are missing the specific scenario (VMs or configuration) required to trigger this one.

Have you been able to find any further information in the host logs?

If not, could you export and send us a status report using xen-bugtool? Perhaps one of our developers will be able to spot something in it that helps explain the issue.

[jr-m4]

3 pools updated. RPU went fine, where one is HA. A couple of test-backups went fine as well.

[robertblissitt]

Greetings, @majorp93 . I have always heard that Dynamic Memory should not be used in XenServer/XCP-ng as it is not 100% stable, and that has been my experience as well.

[lmolina]

We have just released a security update for a critical Linux kernel vulnerability CVE-2026-98432 / VSA-2026-056.

Full details are available on the blog: https://xcp-ng.org/blog/2026/11/04/november-2026-security-updates-2-for-xcp-ng-8-3-lts/

[manilx]

@lmolina And you couldn't have waited yesterday's update for ONE DAY!

Give me a break, another 1hr updating.....

It's GREAT that you address security updates.... BUT come on.

[gduperrey]

@manilx

If we could have waited, we would have. However, the xen security update released yesterday, given its critical nature, did not allow for that.

Please rest assured that we are aware of this issue and are doing our utmost to keep the number of releases to a minimum. We are also discussing how to communicate regarding the most critical updates and how to limit the need for reboots.

[acebmxer]

applying the updates on hosts at work. So far 2 out of 3 pools updated succesfully. however when i go to attempt to apply updates on the last pool i see this...

[ThierryC01]

@lmolina When will we be able to update or patch everything from XO6? It's kind of a bummer to switch from a slick UI to the legacy, yet functional one, just to apply some patches.