Hello @codemercenary In theory Xen is NUMA aware (See https://wiki.xenproject.org/wiki/Xen_on_NUMA_Machines) so you would leave memory interleaving option of the BIOS off so that Xen does what it thinks is necessary. But these kinds of settings are very very workload sensitive and in fact it's usually a good idea to just test what works best for you. Regards, Yann

Hi, I just wanted to comment that the provided packages work for all my server. Thank you!

Just to close the loop on this: XO 6.4 (released on April 29) includes a specific fix for VNC reconnection, which may have been causing keyboard focus loss and, in turn, modifier keys to get stuck (PR #9727). At least, that’s my understanding; I could be completely wrong, so please take it with a grain of salt. If you’re on XO 6 and still seeing the issue, upgrading to 6.4 may help.

Good news, or at least, that's my understanding. This was a known bug and it's fixed in XO 6.4 (released April 29, PR #9681). Upgrading should make those alerts disappear for good. Thanks for the video, it made the report unambiguous.

Very nice job @acebmxer Well done

I have completed the latest patches that came out and yet i still face this same issue.. anyone have a solution for this?? id "0moke6b1h-u49q39cfnql" properties method "vm.migrate" params vm "038b1fb8-e86c-9169-2ebc-c18b4e9c53f7" migrationNetwork "9a74000a-8481-1b36-2dd1-a58cd6747623" targetHost "82cf5c29-db31-4cc3-b057-db8f85324232" name "API call: vm.migrate" userId "02d261f6-840b-4148-9eb5-d181432af7d2" type "api.call" start 1777487518565 status "failure" updatedAt 1777487530207 warnings 0 data pif "c13bcf0c-ef0c-27bd-592a-58ef4eedf18d" message "PIF is not attached" end 1777487530207 result code "INTERNAL_ERROR" params 0 'xenopsd internal error: Xmlrpc.Parse_error("open_tag(html)", "open_tag(methodResponse)", _)' task uuid "bd7f8d2a-6d91-3d6a-0da2-e4c1c3758346" name_label "Async.VM.pool_migrate" name_description "" allowed_operations [] current_operations {} created "20260429T18:31:58Z" finished "20260429T18:32:09Z" status "failure" resident_on "OpaqueRef:6dadebb7-39c8-4bb7-a559-909a80824227" progress 1 type "<none/>" result "" error_info 0 "INTERNAL_ERROR" 1 'xenopsd internal error: Xmlrpc.Parse_error("open_tag(html)", "open_tag(methodResponse)", _)' other_config {} subtask_of "OpaqueRef:NULL" subtasks [] backtrace "(((process xenopsd-xc)(filename src/lib/xmlrpc.ml)(line 182))((process xenopsd-xc)(filename ocaml/xapi-idl/lib/xcp_client.ml)(line 138))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xenopsd-xc)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xenopsd-xc)(filename ocaml/libs/stunnel/stunnel.ml)(line 412))((process xenopsd-xc)(filename src/lib/idl.ml)(line 550))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 2816))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 103))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_task.ml)(line 111))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3408))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3418))((process xenopsd-xc)(filename ocaml/xenopsd/lib/xenops_server.ml)(line 3439))((process xenopsd-xc)(filename ocaml/xapi-idl/lib/task_server.ml)(line 192))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3479))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3650))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 262))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 268))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 293))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 434))((process xapi)(filename ocaml/xapi/xapi_xenops.ml)(line 3488))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 458))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 141))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 2515))((process xapi)(filename ocaml/xapi/rbac.ml)(line 228))((process xapi)(filename ocaml/xapi/rbac.ml)(line 238))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))" message 'INTERNAL_ERROR(xenopsd internal error: Xmlrpc.Parse_error("open_tag(html)", "open_tag(methodResponse)", _))' name "XapiError" stack 'XapiError: INTERNAL_ERROR(xenopsd internal error: Xmlrpc.Parse_error("open_tag(html)", "open_tag(methodResponse)", _))\n at XapiError.wrap (file:///opt/xo/xo-builds/xen-orchestra-202604291609/packages/xen-api/_XapiError.mjs:16:12)\n at default (file:///opt/xo/xo-builds/xen-orchestra-202604291609/packages/xen-api/_getTaskResult.mjs:13:29)\n at Xapi._addRecordToCache (file:///opt/xo/xo-builds/xen-orchestra-202604291609/packages/xen-api/index.mjs:1078:24)\n at file:///opt/xo/xo-builds/xen-orchestra-202604291609/packages/xen-api/index.mjs:1112:14\n at Array.forEach (<anonymous>)\n at Xapi._processEvents (file:///opt/xo/xo-builds/xen-orchestra-202604291609/packages/xen-api/index.mjs:1102:12)\n at Xapi._watchEvents (file:///opt/xo/xo-builds/xen-orchestra-202604291609/packages/xen-api/index.mjs:1275:14)' i had to manually migrate VM's using the xe command to do the patching and even now to move hosts back. So the loadbalancing or anything is not working

@bnerickson said: Thanks for the reminder, for some reason it was not seen before (all applogies). I'm unsure where to report this, Here or in related open projects directly (assuming you know where to look at) but let me guide you. but syslog reports the following warning when a VM starts: (UDEV-WORKER) cpu0: Process '/bin/sh -c '[ -e /dev/xen/xenbus ] && [ -e /sys/devices/system/cpu/cpu0/online ] && echo 1 > /sys/devices/system/cpu/cpu0/online'' failed with exit code 1. Good catch, it's a bad habit to use AND (&&) in conditionals. Hardcore programmers prefer to use OR (||) in scripts, this practice prevents failed exit and make debugging easier (with set -xe). (...) > ACTION=="add", SUBSYSTEM=="cpu", RUN+="/bin/sh -c 'if [ -e /dev/xen/xenbus ] && [ -e /sys$devpath/online ]; then echo 1 > /sys$devpath/online; fi'" That would work, What about this "simpler" form ? [ ! -e /dev/xen/xenbus ] || [ ! -e /sys$devpath/online ] || echo 1 > /sys$devpath/online Could be harder to read, but I get used to. If you want you can contribute the fix directly to upstream since the file is public at: https://github.com/xenserver/xe-guest-utilities/blob/master/mk/xen-vcpu-hotplug.rules About the exec bit, here is the line that should be changed: https://github.com/xcp-ng-rpms/xcp-ng-pv-tools/commit/3d134fbd0f3ac9e2f3cfa914e38b33529635d458#r183816712 @bnerickson, If you need any mentoring for OSS contributions, I would be more than a happy to help. Since it's a lowest priority you can take all the time you need, and If too busy, no problem we can do it for you and then backport the change to XCP-ng. 1 stormi committed to xcp-ng-rpms/xcp-ng-pv-tools Build the tools from the sources - Build 32 bit and 64 bit xe-guest-utilities - Build TGZ, RPM and DEB packages for xe-guest-utilities - Put the result into the ISO and the ISO into the xcp-ng-pv-tools RPM (as before). It's a bit hacky, but as the binaries are statically linked, this peculiar build process should work.