Hello,

I plan to install my XOSTOR cluster on a pool of 7 nodes with 3 replicas, but not all nodes at once because disks are in use.
consider:

• node1
• node2
• node ...
• node 5
• node 6
• node 7.

with 2 disks on each

• sda: 128GB for the OS
• sdb: 1TB for local sr ( for now )

I emptied node 6 & 7.

so, here is what i plan to do:

• On ALL NODES: setup linstor packages

Run the install script on node 6 & 7 to add their disks
so:

node6# install.sh --disks /dev/sdb
node7# install.sh --disks /dev/sdb

Then, configure the SR and the linstor plugin manager as the following

xe sr-create \ 
    type=linstor name-label=pool-01 \
    host-uuid=XXXX \
    device-config:group-name=linstor_group/thin_device device-config:redundancy=3 shared=true device-config:provisioning=thin

Normally, i should have a linstor cluster running of 2 nodes ( 2 satellite and one controller randomly placed ) with only 2 disks and then, only 2/3 working replicas.

The cluster SHOULD be usable ( i'm right on this point ? )

The next step, would be to move VM from node 5 on it to evacuate node 5. and then add it to the cluster by the following

node5# install.sh --disks /dev/sdb
node5# xe host-call-plugin \
  host-uuid=node5-uuid \
  plugin=linstor-manager \
  fn=addHost args:groupName=linstor_group/thin_device

That should deploy satelite on node 5 and add the disk.

I normally should have 3/3 working replicas and can start to deploy others nodes progressively.

I'm right on the process ?

aS mentionned in the discord, i will post my feedbacks and results from my setup once i finalized it. ( maybe thought a blog post somewhere ).

Thanks to provide xostor in opensource, it's clearly the missing piece for this virtualization stack in opensource ( vs proxmox )

Hello, @TheNorthernLight

XOStor is now available in 8.3 since it's LTS

@olivierlambert our opnsense resets the TCP states so the firewall block packet because it forgot about the tcp session.

And then, a timeout occured in the middle of the export.

Hello @olivierlambert

I confirm my issue came from my Firewall so, not related to XO.

However, it could be great to make logs more "clear", i mean:

Error: read ETIMEDOUT" 

Become

Error: read ETIMEDOUT while connect to X.X.X.X:ABC

That would permit to understand more quickly my "real and weird" issue

Best regards,

Hello, @olivierlambert

Latest version,

XCP-NG 8.3 latest version, i'm trying to perform disaster recovery backup,

I think the problems seems to come from my firewall which "cut" the TCP session.

@henri9813 I found this in my logs don't know if it's related

Apr 30 21:24:00 gco-002-rbx-002 xapi: [debug||8618534 HTTPS 10.255.0.250->:::80|put_import|import] import handler
Apr 30 21:24:00 gco-002-rbx-002 xapi: [debug||8618535 /var/lib/xcp/xapi|post_root|dummytaskhelper] task dispatch:session.slave_login D:bd14de344b0e created by task D:3fcf04a3282e
Apr 30 21:24:00 gco-002-rbx-002 xapi: [ info||8618535 /var/lib/xcp/xapi|session.slave_login D:fefe31620afb|xapi_session] Session.create trackid=7ded2f51c51fbf82e2c5ac7cee9931dc pool=true uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618533 ||xapi_compression] nice failed to decompress: exit code 1
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] Raised End_of_file
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 1/13 xapi Raised at file unix/tar_unix.ml, line 34
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 2/13 xapi Called from file unix/tar_unix.ml, line 26
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 3/13 xapi Called from file lib/tar.ml, line 829
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 4/13 xapi Called from file lib/tar.ml, line 660
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 5/13 xapi Called from file lib/tar.ml, line 666
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 6/13 xapi Called from file lib/tar.ml, line 673
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 7/13 xapi Called from file lib/tar.ml, line 749
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 8/13 xapi Called from file lib/tar.ml, line 867
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 9/13 xapi Called from file ocaml/xapi/import.ml, line 2132
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 10/13 xapi Called from file ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 11/13 xapi Called from file ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml, line 39
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 12/13 xapi Called from file ocaml/xapi/import.ml, line 2554
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace] 13/13 xapi Called from file lib/backtrace.ml, line 177
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|backtrace]
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|VM.import R:9d05d125c060|import] Prematurely reached end-of-file during import
Apr 30 21:24:00 gco-002-rbx-002 xapi: [debug||8618536 /var/lib/xcp/xapi|post_root|dummytaskhelper] task dispatch:pool.get_all D:8b21a6838cb8 created by task D:fefe31620afb
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] VM.import R:9d05d125c060 failed with exception Server_error(IMPORT_ERROR_PREMATURE_EOF, [  ])
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] Raised Server_error(IMPORT_ERROR_PREMATURE_EOF, [  ])
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] 1/7 xapi Raised at file lib/backtrace.ml, line 210
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] 2/7 xapi Called from file ocaml/xapi/import.ml, line 2204
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] 3/7 xapi Called from file ocaml/xapi/server_helpers.ml, line 72
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] 4/7 xapi Called from file ocaml/xapi/server_helpers.ml, line 94
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] 5/7 xapi Called from file ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml, line 24
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] 6/7 xapi Called from file ocaml/libs/xapi-stdext/lib/xapi-stdext-pervasives/pervasiveext.ml, line 39
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace] 7/7 xapi Called from file ocaml/libs/log/debug.ml, line 250
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 :::80|VM.import D:7c2fd25192d0|backtrace]
Apr 30 21:24:00 gco-002-rbx-002 xapi: [debug||8618537 /var/lib/xcp/xapi|post_root|dummytaskhelper] task dispatch:session.logout D:f7e71c366389 created by task D:7c2fd25192d0
Apr 30 21:24:00 gco-002-rbx-002 xapi: [ info||8618537 /var/lib/xcp/xapi|session.logout D:218a803a945b|xapi_session] Session.destroy trackid=b2e981e2a3cbc9fc91324f34ef240f5d
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|put_import|backtrace] VM.import D:7c2fd25192d0 failed with exception Server_error(IMPORT_ERROR_PREMATURE_EOF, [  ])
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|put_import|backtrace] Raised Server_error(IMPORT_ERROR_PREMATURE_EOF, [  ])
Apr 30 21:24:00 gco-002-rbx-002 xapi: [error||8618530 HTTPS 10.255.0.250->:::80|put_import|backtrace] 1/14 xapi Raised at file ocaml/libs/log/debug.ml, line 267

Hello,

I have this error in xen-orchestra.

Currently, my xen orchestra is in a different network than my hypervisors.

My backup target is in another DC of course thought NFS ( via Wireguard VPN ).

Usually, it's doens't cause issue, but now, i have this error

xen-orchestra  | 2025-04-30T16:01:03.955Z xo:backups:AbstractVmRunner WARN writer step failed {
xen-orchestra  |   error: Error: read ETIMEDOUT
xen-orchestra  |       at TLSWrap.onStreamRead (node:internal/stream_base_commons:218:20)
xen-orchestra  |       at TLSWrap.callbackTrampoline (node:internal/async_hooks:130:17) {
xen-orchestra  |     errno: -110,
xen-orchestra  |     code: 'ETIMEDOUT',
xen-orchestra  |     syscall: 'read'
xen-orchestra  |   },
xen-orchestra  |   step: 'writer.transfer()',
xen-orchestra  |   writer: 'IncrementalRemoteWriter'
xen-orchestra  | }

It's problematic because, here, i think the issue is between Xen-orchestra and the final hypervisor, right ?

( it could be great to add some details on the error to ease the diagnostics )

thanks !

Thanks !

Hello,

I created some VMs using fast clone.

I also saw on forums, GitHub and documentation that It could impact on IO performance for the VM.

How could I "unlink" the VM from the template vm ?

Best regards,

Hello,

Thanks for your answer @nikade

I migrate both storage and vm.

I have 25Gb/s nic, but I have a 5Gb/s limitation at switch level.

But I found on another topic an explanation about this, this could be related to SMAPIv1.

Best regards

Hello,

So, does someone know why I can't go over 145MB/s and how to increase the vdi speed transfert ?

( I have 5Gb/s between nodes )

Thanks !

@Danp Okay, you're a genius, I upgrade this morning after my backup.

So that could explain my issue.

the mentioned thread is exactly my issue, but I didn't find it when I was searching about my issue.

Thanks for all !

@Danp i have 17 commits lates

I'm running https://github.com/vatesfr/xen-orchestra/commit/f9220cd272674ef2f0403c9b784c955783fae334 ) ( 12 July )

If it's XO related, why in xcpng I also see the blocked operation ?

Hello,

I try to migrate a VM between two host but I got:

vm.migrate
{
  "vm": "bca3bd58-8455-d07f-c2a9-ba45daeb6fb4",
  "sr": "b4313d56-d226-c453-cef7-d7180a3daab1",
  "targetHost": "e7e336d6-07b2-4c23-ac95-2a3f7bfa47ab"
}
{
  "code": "OPERATION_BLOCKED",
  "params": [
    "OpaqueRef:f032a458-e996-441b-a352-7e76fdc8dfe7",
    "VM migration is blocked during backup"
  ],
  "task": {
    "uuid": "52474c0f-fcd1-21dd-1a02-5139e0a48289",
    "name_label": "Async.VM.migrate_send",
    "name_description": "",
    "allowed_operations": [],
    "current_operations": {},
    "created": "20240719T12:16:13Z",
    "finished": "20240719T12:16:13Z",
    "status": "failure",
    "resident_on": "OpaqueRef:59f0a1ab-ffe4-4c2b-8465-72f32e9ef71b",
    "progress": 1,
    "type": "<none/>",
    "result": "",
    "error_info": [
      "OPERATION_BLOCKED",
      "OpaqueRef:f032a458-e996-441b-a352-7e76fdc8dfe7",
      "VM migration is blocked during backup"
    ],
    "other_config": {},
    "subtask_of": "OpaqueRef:NULL",
    "subtasks": [],
    "backtrace": "(((process xapi)(filename ocaml/xapi/xapi_vm_lifecycle.ml)(line 731))((process xapi)(filename ocaml/xapi/xapi_vm_helpers.ml)(line 1501))((process xapi)(filename lib/xapi-stdext-pervasives/pervasiveext.ml)(line 24))((process xapi)(filename ocaml/xapi/rbac.ml)(line 205))((process xapi)(filename ocaml/xapi/server_helpers.ml)(line 95)))"
  },

Indeed, in can see this from xe cli:

[14:15 cloud-01-rbx-002 ~]# xe vm-param-get uuid=bca3bd58-8455-d07f-c2a9-ba45daeb6fb4 param-name=blocked-operations
migrate_send: VM migration is blocked during backup; pool_migrate: VM migration is blocked during backup

But ... I have no running operation

[14:19 cloud-01-rbx-002 ~]# xe task-list
[14:19 cloud-01-rbx-002 ~]# 

Do you have any idea ? ( I'm running 8.2.1 ).

Thanks !

Hello,

Could it be possible to allow show graphs in others units than MiB?

Units that could be printed:

• bit/s
• Bytes/s

however, I don't know where this should be configurable on XO

That's stupid, but i spend to much time to realize that was 2Gb/s ( I look the graph 5s and I say to myself, okay, I'm stuck at 200Mb/s, whereas .... ^^'

Best regards,