Posts made by henri9813 | XCP-ng and XO forum

henri9813

Hello there

I start to use slowly XO6 since many features was added.

But i observe this:

I have a VM with 3 backups jobs associated:

02/07/2026 - 8H47: Success
02/07/2026 - 8H48: Fail
02/07/2026 - 8H54: Success

But in the UI, i see this:

The jobs should be order from the "most recent" to "older one" ( or vice versa, i don"t have opinion on it ).

But showing the fail as the first make us thinks that the last jobs fail, that's not the case.

I should have:

Thank you

henri9813

Hello,

All is in the title

Is this normal ?

Thanks !

henri9813

Hello,

Thanks for all !

henri9813

Hello,

To gave more details abouit my case:

XCPNG: 8.3.0
CPU: AMD EPYC 4464P 12-Core Processor
Kernel: 6.12.0-211.18.1.el10_2.x86_64

henri9813

Hello,

When i boot a Rocky Linux 10 with the latest kernel, it's very slow on cpu XXX spinlock event irq.

Someone have the same issue ?

Thanks !

henri9813

@gregoire said:

@olivierlambert I added this feature request in the backlog regarding RPU improvements.

Hello,

Thanks all !

Totally agree with @poddingue , be able to exclude VM which has:

PCI attached devices
Local storage ( maybe ? )

Could be a great option !

henri9813

Hello @McHenry .

Yes but no, once the snapshot is exported, the previous one must be cleaned on local.

Best regards,

henri9813

Hello,

I see also this behavior which is "new" since few weeks.

Previously, when a backup start:

it stake a snapshot ( if there another one before, it delete it ).
it upload the snapshot as a backup
it coalesce the backup on the remote.
end of the game.

Now, the old snapshots are not deleted anymore which can lead easily to some disk full.

Even with a retention of 1, the problem is present.

I observe this only in Backup job, not DR/CR job.

I just updated my XO to latest version, i will see if the issue is fixed.

henri9813

Hello,

In the post about XCP-NG 9.0, there was a discussion about Storage Domain Virtual machines, as i understood, the project was abandonned, but the concept is still doable without this implementation.

https://xcp-ng.org/forum/post/102311

On each of my hosts, i have a VM dedicated for ceph which mount the PCI devices ( NVME disks ).

Everything works like a charm !

But the RPU is not available due to the VMS which have PCI attached resources.

Is it possible to exclude VMS from RPU and make it shutdown before the upgrade ?

something like you done to exclude Security warning about old windows guest tools.

This would be very great

henri9813

Hello @nikade.

I agree but this is my case.

Try to migrate a running VM: error 500
Try to migrate an halted VM: error 500.
Warm migrate: It's okay.

I don't understand myself the difference except it doesn't transfer the "VM" but recreate the VM and import the VDI, ( so , the same things ), but there may be a light difference. I don't know how the "warm migration" works under the hood

henri9813

Hello,

I was able to perform "Warm" migrate from either the slave or the master.

Yes the master was rebooted.

@henri9813 Did you reboot the master after it was updated? If yes, I think you should be able to migrate back the VM's to the master, and then continue patcting the rest of the hosts.

No, i would migrate back vm from MASTER (updated ) to slave ( not updated ), but wasn't working.

Only warm migrations works.

henri9813

Hello, @Danp

My pool is in upgrade, not all nodes are updated.

I evacuate the master hosts's vms to the slave node except 3-4 vms which are "very not important and can be shut for few minutes
I upgrade the master node with success
I WANT to move all vms to the master, but it deosn't have enought disk space, so i tried the following:
a. Migrate the very not important VM to the slave node ( not updated ) to have enought space to move the "important" vms.
b. Move vms of the updated master to ANOTHER pool.

I tried both:

VM running
VM halted

Thanks !

henri9813

I have an updated pool.

I'm evacuating an host for update to another pool.

But the operation fail, and remains in the XO tasks

VM metadata import (on new-hypervisor) 0%

and in the VM.migrate api call detail.

vm.migrate
{
  "vm": "30bb4942-c7fd-a3b3-2690-ae6152d272c5",
  "mapVifsNetworks": {
    "7e1ad49f-d4df-d9d7-2a74-0d00486ae5ff": "b3204067-a3fd-bd19-7214-7856e637d076"
  },
  "migrationNetwork": "e31e7aea-37de-2819-83fe-01bd33509855",
  "sr": "3070cc36-b869-a51f-38ee-bd5de5e4cb6c",
  "targetHost": "36a07da2-7493-454d-836d-df8ada5b958f"
}
{
  "code": "INTERNAL_ERROR",
  "params": [
    "Http_client.Http_error(\"500\", \"{ frame = false; method = GET; uri = /export_metadata?export_snapshots=true&ref=OpaqueRef:75e166f7-5056-a662-f7ff-25c09aee5bec; query = [  ]; content_length = [  ]; transfer encoding = ; version = 1.0; cookie = [ (value filtered) ]; task = ; subtask_of = OpaqueRef:9976b5f2-3381-e79e-a6dd-0c7a20621501; content-type = ; host = ; user_agent = xapi/25.33; }\")"
  ],
  "task": {
    "uuid": "9c87e615-5dca-c714-0c55-5da571ad8fa5",
    "name_label": "Async.VM.assert_can_migrate",
    "name_description": "",
    "allowed_operations": [],
    "current_operations": {},
    "created": "20260131T08:16:14Z",
    "finished": "20260131T08:16:14Z",
    "status": "failure",
    "resident_on": "OpaqueRef:37858c1b-fa8c-5733-ed66-dcd4fc7ae88c",
    "progress": 1,
    "type": "<none/>",
    "result": "",
    "error_info": [
      "INTERNAL_ERROR",
      "Http_client.Http_error(\"500\", \"{ frame = false; method = GET; uri = /export_metadata?export_snapshots=true&ref=OpaqueRef:75e166f7-5056-a662-f7ff-25c09aee5bec; query = [  ]; content_length = [  ]; transfer encoding = ; version = 1.0; cookie = [ (value filtered) ]; task = ; subtask_of = OpaqueRef:9976b5f2-3381-e79e-a6dd-0c7a20621501; content-type = ; host = ; user_agent = xapi/25.33; }\")"
    ],
    "other_config": {},
    "subtask_of": "OpaqueRef:NULL",
    "subtasks": [],
    "backtrace": "(((process xapi)(filename ocaml/libs/http-lib/http_client.ml)(line 215))((process xapi)(filename ocaml/libs/http-lib/http_client.ml)(line 228))((process xapi)(filename ocaml/libs/http-lib/xmlrpc_client.ml)(line 375))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/importexport.ml)(line 313))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml)(line 39))((process xapi)(filename ocaml/xapi/xapi_vm_migrate.ml)(line 1920))((process xapi)(filename ocaml/xapi/message_forwarding.ml)(line 2551))((process xapi)(filename ocaml/xapi/rbac.ml)(line 229))((process xapi)(filename ocaml/xapi/rbac.ml)(line 239))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 78)))"
  },
  "message": "INTERNAL_ERROR(Http_client.Http_error(\"500\", \"{ frame = false; method = GET; uri = /export_metadata?export_snapshots=true&ref=OpaqueRef:75e166f7-5056-a662-f7ff-25c09aee5bec; query = [  ]; content_length = [  ]; transfer encoding = ; version = 1.0; cookie = [ (value filtered) ]; task = ; subtask_of = OpaqueRef:9976b5f2-3381-e79e-a6dd-0c7a20621501; content-type = ; host = ; user_agent = xapi/25.33; }\"))",
  "name": "XapiError",
  "stack": "XapiError: INTERNAL_ERROR(Http_client.Http_error(\"500\", \"{ frame = false; method = GET; uri = /export_metadata?export_snapshots=true&ref=OpaqueRef:75e166f7-5056-a662-f7ff-25c09aee5bec; query = [  ]; content_length = [  ]; transfer encoding = ; version = 1.0; cookie = [ (value filtered) ]; task = ; subtask_of = OpaqueRef:9976b5f2-3381-e79e-a6dd-0c7a20621501; content-type = ; host = ; user_agent = xapi/25.33; }\"))
    at XapiError.wrap (file:///etc/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
    at default (file:///etc/xen-orchestra/packages/xen-api/_getTaskResult.mjs:13:29)
    at Xapi._addRecordToCache (file:///etc/xen-orchestra/packages/xen-api/index.mjs:1078:24)
    at file:///etc/xen-orchestra/packages/xen-api/index.mjs:1112:14
    at Array.forEach (<anonymous>)
    at Xapi._processEvents (file:///etc/xen-orchestra/packages/xen-api/index.mjs:1102:12)
    at Xapi._watchEvents (file:///etc/xen-orchestra/packages/xen-api/index.mjs:1275:14)"
}

My XO is up to date.

I already update the master, i'm processing the slave.

But neither the master ( updated ) nor the slave ( not updated ) can migrate VM to an updated pool.

Do you have an idea ?

henri9813

Hello @florent ,

You mean, "check" right ? because if it's doesn't check, the job is finished before the merge and the merge run in background in xen orchestra.

Best regards,

henri9813

Hello, @Pilow

Thanks for the tips !

But it would be great to see it in the Task UI to find more easily the trick.

henri9813

Hello,

Sometimes, i have on some jobs ( which run every 4 hours ).

Howerver, the previous job is finished in success

Is it possible that has a relation with the mergeWorker of backups which could be running ? if it doesn't finished his operations ?

Example of logs:

xen-orchestra      | 2025-11-27T00:44:43.487Z xo:backups:mergeWorker INFO merge in progress {
xen-orchestra      |   done: 2057,
xen-orchestra      |   parent: '/xo-vm-backups/f37e259d-beaa-7617-e6f1-be814f21e056/vdis/29e0185e-2f67-44d4-bb9e-ee2a772e2543/b09c0230-219f-4ddf-8e19-bfed1464014f/20251126T070610Z.vhd',
xen-orchestra      |   progress: 25,
xen-orchestra      |   total: 8128
xen-orchestra      | }

Is it possible to have this in the XO Tasks sections ? it's interessant to see this.

Best regards

henri9813

Hello @ronan-a ,

but how recover from this situation ?

Thanks !

henri9813

Hello,

Thanks for your work !

We have some hypervisors of tests at Gladhost, we can use them with pleasure to test your work on xcp-ng 8.3 !

Best regards

henri9813

Hello,

I got my whole xostor destroyed, i don't know how precisely.

I found some errors in sattelite

Error context:
        An error occurred while processing resource 'Node: 'host', Rsc: 'xcp-volume-e011c043-8751-45e6-be06-4ce9f8807cad''
ErrorContext:
  Details:     Command 'lvcreate --config 'devices { filter=['"'"'a|/dev/md127|'"'"','"'"'a|/dev/md126p3|'"'"','"'"'r|.*|'"'"'] }' --virtualsize 52543488k linstor_primary --thinpool thin_device --name xcp-volume-e011c043-8751-45e6-be06-4ce9f8807cad_00000' returned with exitcode 5. 

Standard out: 


Error message: 
  WARNING: Remaining free space in metadata of thin pool linstor_primary/thin_device is too low (98.06% >= 96.30%). Resize is recommended.
  Cannot create new thin volume, free space in thin pool linstor_primary/thin_device reached threshold.

of course, i checked, my SR was not full

And the controller crashed, and i couldn't make it works.

Here is the error i got

==========

Category:                           RuntimeException
Class name:                         IllegalStateException
Class canonical name:               java.lang.IllegalStateException
Generated at:                       Method 'newIllegalStateException', Source file 'DataUtils.java', Line #870

Error message:                      Reading from nio:/var/lib/linstor/linstordb.mv.db failed; file length 2293760 read length 384 at 2445540 [1.4.197/1]

So i deduce the database was fucked-up, i tried to open the file as explained in the documentation, but the linstor schema was "not found" in the file, event if using cat i see data about it.

for now, i leave xostor and i'm back to localstorage until we know what to do when this issue occured with a "solution path".

henri9813

Hello, @DustinB

The https://vates.tech/xostor/ says:

The maximum size of any single Virtual Disk Image (VDI) will always be limited by the smallest disk in your cluster.

But in this case, maybe it can be stored in the "2TB disks" ? Maybe others can answer, i didn't test it.