Hello, @burbilog
I think theses backup doesn't come from your backup plan.
Is there maybe snapshots on the VM ? 2022 is "very old"
Your full backups include memory which is not "classical" in a incremental backups.
Offline
Posts
-
RE: Full or not?
-
RE: Unable to add new node to pool using XOSTOR
Hello,
I tried on a new pool.
a little different scenario since i don't create xostor for now, on my previous example, i tried to add a node as replacement of an existing one..
I just run the install script only on node 1.
When i try make node2 join the pool, i reproduce the incompatible sm error i got previously.
The things which is "bizarre", is i don't have the license issue i got on Xen-orchestra. ( maybe it was finally not related ? )
Here is the complete logs.
pool.mergeInto { "sources": [ "17510fe0-db23-9414-f3df-2941bd34f8dc" ], "target": "cc91fcdc-c7a8-a44c-65b3-a76dced49252", "force": true } { "code": "POOL_JOINING_SM_FEATURES_INCOMPATIBLE", "params": [ "OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd", "" ], "call": { "duration": 1061, "method": "pool.join_force", "params": [ "* session id *", "<MASTER_IP>", "root", "* obfuscated *" ] }, "message": "POOL_JOINING_SM_FEATURES_INCOMPATIBLE(OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd, )", "name": "XapiError", "stack": "XapiError: POOL_JOINING_SM_FEATURES_INCOMPATIBLE(OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd, ) at Function.wrap (file:///etc/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12) at file:///etc/xen-orchestra/packages/xen-api/transports/json-rpc.mjs:38:21 at runNextTicks (node:internal/process/task_queues:60:5) at processImmediate (node:internal/timers:454:9) at process.callbackTrampoline (node:internal/async_hooks:130:17)" }``` -
Multiple disks groups
Hello,
we have nodes with multiple disk group.
Does someone experienced multiple xostor on different local group ?
In our case
- 2x1TB & 2x2TB
It should works easily because we gonna register 2 xostor, but maybe should we put all disks in the same VG ? but in this case, the max vdi size will be lower ?
I purpose to edit the install script to provide group name selection to permit this use case easily.
It's ok ?
-
RE: Unable to add new node to pool using XOSTOR
Hello @ronan-a
Just to be sure, you want logs of the node which want to join or the master ?
Bonne journée
-
RE: Backup fail whereas xostor cluster is "healthy"
Hello @ronan-a
I will reproduce the case, i will re-destroy one hypervisor and retrigger the case.
Thank you @ronan-a et @olivierlambert
If you need me to tests some special case don't hesit, we have a pool dedicated for this
-
RE: Unable to add new node to pool using XOSTOR
Hello, @ronan-a
I will reinstall my hypervisor this week.
I will reproduce it and then, resend you the logs.
Bonne journée,
-
RE: Unable to add new node to pool using XOSTOR
In this case @olivierlambert
The new node doesn't have any linstor version installed before.
I planned to install it AFTER adding it to a pool, but that's not possible. It's a "dependency" to join the pool. ( as i understand ^^ )
If my interpretation and solution is correct, can i propose a MR to add it thought the documentation ? ( for the CLI part at least, the xo side looks to be a bug for me )
-
RE: Unable to add new node to pool using XOSTOR
Hello @olivierlambert
All my hosts were up to date ( verified many times ).
But once the linstor installed, the error disappeared.
Here is the my ssh i keeped open
( thanks Mac )[00:55 node-3 ~]# xe pool-join master-address=XXXXX master-username=XXXX master-password="YYYYY" The host joining the pool has an incompatible set of sm features from the pool coordinator. Make sure the sm are of the same versions and try again. pool_sm_ref: OpaqueRef:4d762998-79ee-8e88-1f0f-5b74aac9af85 candidate_sm_ref: [00:56 node-3 ~]# ^C [00:58 node-3 ~]# yum install -y xcp-ng-release-linstor Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Excluding mirror: updates.xcp-ng.org * xcp-ng-base: mirrors.xcp-ng.org Excluding mirror: updates.xcp-ng.org * xcp-ng-updates: mirrors.xcp-ng.org Resolving Dependencies --> Running transaction check ---> Package xcp-ng-release-linstor.noarch 0:1.4-2.xcpng8.3 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================ Package Arch Version Repository Size ============================================================================================================================================================ Installing: xcp-ng-release-linstor noarch 1.4-2.xcpng8.3 xcp-ng-updates 3.9 k Transaction Summary ============================================================================================================================================================ Install 1 Package Total download size: 3.9 k Installed size: 477 Downloading packages: xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch.rpm | 3.9 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch 1/1 Verifying : xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch 1/1 Installed: xcp-ng-release-linstor.noarch 0:1.4-2.xcpng8.3 Complete! [00:58 node-3 ~]# yum install -y xcp-ng-linstor Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Excluding mirror: updates.xcp-ng.org * xcp-ng-base: mirrors.xcp-ng.org Excluding mirror: updates.xcp-ng.org * xcp-ng-updates: mirrors.xcp-ng.org xcp-ng-linstor/signature | 473 B 00:00:00 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-xcpng Importing GPG key 0x3FD3AC9E: Userid : "XCP-ng Key (XCP-ng Official Signing Key) <security@xcp-ng.org>" Fingerprint: 34ac 2eb6 8248 fed6 d076 838a cd75 783a 3fd3 ac9e Package : xcp-ng-release-8.3.0-32.x86_64 (@xcp:main/$releasever) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-xcpng xcp-ng-linstor/signature | 3.0 kB 00:00:00 !!! xcp-ng-linstor/primary_db | 26 kB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package xcp-ng-linstor.noarch 0:1.2-2.xcpng8.3 will be installed --> Processing Dependency: drbd for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: drbd-reactor for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: http-nbd-transfer for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: kmod-drbd for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: linstor-client for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: linstor-controller for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: linstor-satellite for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Running transaction check ---> Package drbd.x86_64 0:9.28.0-1.el7 will be installed --> Processing Dependency: drbd-bash-completion = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 --> Processing Dependency: drbd-pacemaker = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 --> Processing Dependency: drbd-udev = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 --> Processing Dependency: drbd-utils = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 --> Processing Dependency: drbd-xen = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 ---> Package drbd-reactor.x86_64 0:1.4.1-1 will be installed ---> Package http-nbd-transfer.x86_64 0:1.6.0-1.xcpng8.3 will be installed --> Processing Dependency: nbdkit for package: http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64 ---> Package kmod-drbd.x86_64 0:9.2.11-1.1.xcpng8.3 will be installed ---> Package linstor-client.noarch 0:1.23.0-1.xcpng8.3 will be installed --> Processing Dependency: python-linstor >= 1.23.0 for package: linstor-client-1.23.0-1.xcpng8.3.noarch ---> Package linstor-controller.noarch 0:1.29.0-1.el7_9 will be installed --> Processing Dependency: linstor-common = 1.29.0 for package: linstor-controller-1.29.0-1.el7_9.noarch --> Processing Dependency: jre-11-headless for package: linstor-controller-1.29.0-1.el7_9.noarch ---> Package linstor-satellite.noarch 0:1.29.0-1.el7_9 will be installed --> Running transaction check ---> Package drbd-bash-completion.x86_64 0:9.28.0-1.el7 will be installed ---> Package drbd-pacemaker.x86_64 0:9.28.0-1.el7 will be installed ---> Package drbd-udev.x86_64 0:9.28.0-1.el7 will be installed ---> Package drbd-utils.x86_64 0:9.28.0-1.el7 will be installed ---> Package drbd-xen.x86_64 0:9.28.0-1.el7 will be installed ---> Package java-11-openjdk-headless.x86_64 1:11.0.23.0.9-2.el7_9 will be installed --> Processing Dependency: copy-jdk-configs >= 3.3 for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: tzdata-java >= 2023d for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: javapackages-tools for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: libasound.so.2(ALSA_0.9)(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: libasound.so.2(ALSA_0.9.0rc4)(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: lksctp-tools(x86-64) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: pcsc-lite-libs(x86-64) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: libasound.so.2()(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: libharfbuzz.so.0()(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 ---> Package linstor-common.noarch 0:1.29.0-1.el7_9 will be installed ---> Package nbdkit.x86_64 0:1.8.0-4.el7 will be installed ---> Package python-linstor.noarch 0:1.23.0-1.xcpng8.3 will be installed --> Running transaction check ---> Package alsa-lib.x86_64 0:1.1.4.1-2.el7 will be installed ---> Package copy-jdk-configs.noarch 0:3.3-10.el7_5 will be installed ---> Package harfbuzz.x86_64 0:1.3.2-1.el7 will be installed --> Processing Dependency: libgraphite2.so.3()(64bit) for package: harfbuzz-1.3.2-1.el7.x86_64 ---> Package javapackages-tools.noarch 0:3.4.1-11.el7 will be installed --> Processing Dependency: python-javapackages = 3.4.1-11.el7 for package: javapackages-tools-3.4.1-11.el7.noarch --> Processing Dependency: libxslt for package: javapackages-tools-3.4.1-11.el7.noarch ---> Package lksctp-tools.x86_64 0:1.0.17-2.el7 will be installed ---> Package pcsc-lite-libs.x86_64 0:1.8.8-8.el7 will be installed ---> Package tzdata-java.noarch 0:2024a-1.el7 will be installed --> Running transaction check ---> Package graphite2.x86_64 0:1.3.10-1.el7_3 will be installed ---> Package libxslt.x86_64 0:1.1.28-5.el7 will be installed ---> Package python-javapackages.noarch 0:3.4.1-11.el7 will be installed --> Processing Dependency: python-lxml for package: python-javapackages-3.4.1-11.el7.noarch --> Running transaction check ---> Package python-lxml.x86_64 0:3.2.1-4.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================ Package Arch Version Repository Size ============================================================================================================================================================ Installing: xcp-ng-linstor noarch 1.2-2.xcpng8.3 xcp-ng-linstor 3.6 k Installing for dependencies: alsa-lib x86_64 1.1.4.1-2.el7 xcp-ng-base 421 k copy-jdk-configs noarch 3.3-10.el7_5 xcp-ng-base 21 k drbd x86_64 9.28.0-1.el7 xcp-ng-linstor 22 k drbd-bash-completion x86_64 9.28.0-1.el7 xcp-ng-linstor 8.5 k drbd-pacemaker x86_64 9.28.0-1.el7 xcp-ng-linstor 47 k drbd-reactor x86_64 1.4.1-1 xcp-ng-linstor 4.3 M drbd-udev x86_64 9.28.0-1.el7 xcp-ng-linstor 7.0 k drbd-utils x86_64 9.28.0-1.el7 xcp-ng-linstor 657 k drbd-xen x86_64 9.28.0-1.el7 xcp-ng-linstor 9.8 k graphite2 x86_64 1.3.10-1.el7_3 xcp-ng-base 115 k harfbuzz x86_64 1.3.2-1.el7 xcp-ng-base 176 k http-nbd-transfer x86_64 1.6.0-1.xcpng8.3 xcp-ng-updates 30 k java-11-openjdk-headless x86_64 1:11.0.23.0.9-2.el7_9 xcp-ng-base 39 M javapackages-tools noarch 3.4.1-11.el7 xcp-ng-base 72 k kmod-drbd x86_64 9.2.11-1.1.xcpng8.3 xcp-ng-linstor 2.7 M libxslt x86_64 1.1.28-5.el7 xcp-ng-base 241 k linstor-client noarch 1.23.0-1.xcpng8.3 xcp-ng-linstor 266 k linstor-common noarch 1.29.0-1.el7_9 xcp-ng-linstor 54 M linstor-controller noarch 1.29.0-1.el7_9 xcp-ng-linstor 14 M linstor-satellite noarch 1.29.0-1.el7_9 xcp-ng-linstor 880 k lksctp-tools x86_64 1.0.17-2.el7 xcp-ng-base 87 k nbdkit x86_64 1.8.0-4.el7 xcp-ng-base 57 k pcsc-lite-libs x86_64 1.8.8-8.el7 xcp-ng-base 34 k python-javapackages noarch 3.4.1-11.el7 xcp-ng-base 31 k python-linstor noarch 1.23.0-1.xcpng8.3 xcp-ng-linstor 163 k python-lxml x86_64 3.2.1-4.el7 xcp-ng-base 758 k tzdata-java noarch 2024a-1.el7 xcp-ng-base 186 k Transaction Summary ============================================================================================================================================================ Install 1 Package (+27 Dependent packages) Total download size: 118 M Installed size: 286 M Downloading packages: (1/28): drbd-9.28.0-1.el7.x86_64.rpm | 22 kB 00:00:00 (2/28): drbd-bash-completion-9.28.0-1.el7.x86_64.rpm | 8.5 kB 00:00:00 (3/28): drbd-pacemaker-9.28.0-1.el7.x86_64.rpm | 47 kB 00:00:00 (4/28): drbd-udev-9.28.0-1.el7.x86_64.rpm | 7.0 kB 00:00:00 (5/28): copy-jdk-configs-3.3-10.el7_5.noarch.rpm | 21 kB 00:00:00 (6/28): drbd-utils-9.28.0-1.el7.x86_64.rpm | 657 kB 00:00:00 (7/28): drbd-xen-9.28.0-1.el7.x86_64.rpm | 9.8 kB 00:00:00 (8/28): drbd-reactor-1.4.1-1.x86_64.rpm | 4.3 MB 00:00:00 (9/28): alsa-lib-1.1.4.1-2.el7.x86_64.rpm | 421 kB 00:00:00 (10/28): graphite2-1.3.10-1.el7_3.x86_64.rpm | 115 kB 00:00:00 (11/28): harfbuzz-1.3.2-1.el7.x86_64.rpm | 176 kB 00:00:00 (12/28): kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64.rpm | 2.7 MB 00:00:00 (13/28): javapackages-tools-3.4.1-11.el7.noarch.rpm | 72 kB 00:00:00 (14/28): linstor-client-1.23.0-1.xcpng8.3.noarch.rpm | 266 kB 00:00:00 (15/28): http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64.rpm | 30 kB 00:00:00 (16/28): libxslt-1.1.28-5.el7.x86_64.rpm | 241 kB 00:00:00 (17/28): linstor-controller-1.29.0-1.el7_9.noarch.rpm | 14 MB 00:00:00 (18/28): java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64.rpm | 39 MB 00:00:00 (19/28): linstor-satellite-1.29.0-1.el7_9.noarch.rpm | 880 kB 00:00:00 (20/28): lksctp-tools-1.0.17-2.el7.x86_64.rpm | 87 kB 00:00:00 (21/28): pcsc-lite-libs-1.8.8-8.el7.x86_64.rpm | 34 kB 00:00:00 (22/28): python-linstor-1.23.0-1.xcpng8.3.noarch.rpm | 163 kB 00:00:00 (23/28): nbdkit-1.8.0-4.el7.x86_64.rpm | 57 kB 00:00:00 (24/28): python-javapackages-3.4.1-11.el7.noarch.rpm | 31 kB 00:00:00 (25/28): xcp-ng-linstor-1.2-2.xcpng8.3.noarch.rpm | 3.6 kB 00:00:00 (26/28): python-lxml-3.2.1-4.el7.x86_64.rpm | 758 kB 00:00:00 (27/28): linstor-common-1.29.0-1.el7_9.noarch.rpm | 54 MB 00:00:00 (28/28): tzdata-java-2024a-1.el7.noarch.rpm | 186 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 88 MB/s | 118 MB 00:00:01 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : drbd-utils-9.28.0-1.el7.x86_64 1/28 Installing : libxslt-1.1.28-5.el7.x86_64 2/28 Installing : tzdata-java-2024a-1.el7.noarch 3/28 Installing : python-lxml-3.2.1-4.el7.x86_64 4/28 Installing : python-javapackages-3.4.1-11.el7.noarch 5/28 Installing : javapackages-tools-3.4.1-11.el7.noarch 6/28 Installing : kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64 7/28 Installing : drbd-pacemaker-9.28.0-1.el7.x86_64 8/28 Installing : drbd-xen-9.28.0-1.el7.x86_64 9/28 Installing : drbd-udev-9.28.0-1.el7.x86_64 10/28 Installing : drbd-bash-completion-9.28.0-1.el7.x86_64 11/28 Installing : drbd-9.28.0-1.el7.x86_64 12/28 Installing : drbd-reactor-1.4.1-1.x86_64 13/28 Installing : alsa-lib-1.1.4.1-2.el7.x86_64 14/28 Installing : nbdkit-1.8.0-4.el7.x86_64 15/28 Installing : http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64 16/28 Installing : pcsc-lite-libs-1.8.8-8.el7.x86_64 17/28 Installing : lksctp-tools-1.0.17-2.el7.x86_64 18/28 Installing : python-linstor-1.23.0-1.xcpng8.3.noarch 19/28 Installing : linstor-client-1.23.0-1.xcpng8.3.noarch 20/28 Installing : copy-jdk-configs-3.3-10.el7_5.noarch 21/28 Installing : graphite2-1.3.10-1.el7_3.x86_64 22/28 Installing : harfbuzz-1.3.2-1.el7.x86_64 23/28 Installing : 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 24/28 Installing : linstor-common-1.29.0-1.el7_9.noarch 25/28 Installing : linstor-satellite-1.29.0-1.el7_9.noarch 26/28 Installing : linstor-controller-1.29.0-1.el7_9.noarch 27/28 Installing : xcp-ng-linstor-1.2-2.xcpng8.3.noarch 28/28 Verifying : drbd-utils-9.28.0-1.el7.x86_64 1/28 Verifying : graphite2-1.3.10-1.el7_3.x86_64 2/28 Verifying : python-lxml-3.2.1-4.el7.x86_64 3/28 Verifying : harfbuzz-1.3.2-1.el7.x86_64 4/28 Verifying : tzdata-java-2024a-1.el7.noarch 5/28 Verifying : copy-jdk-configs-3.3-10.el7_5.noarch 6/28 Verifying : python-javapackages-3.4.1-11.el7.noarch 7/28 Verifying : linstor-common-1.29.0-1.el7_9.noarch 8/28 Verifying : kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64 9/28 Verifying : drbd-pacemaker-9.28.0-1.el7.x86_64 10/28 Verifying : python-linstor-1.23.0-1.xcpng8.3.noarch 11/28 Verifying : 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 12/28 Verifying : lksctp-tools-1.0.17-2.el7.x86_64 13/28 Verifying : http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64 14/28 Verifying : linstor-client-1.23.0-1.xcpng8.3.noarch 15/28 Verifying : libxslt-1.1.28-5.el7.x86_64 16/28 Verifying : linstor-satellite-1.29.0-1.el7_9.noarch 17/28 Verifying : pcsc-lite-libs-1.8.8-8.el7.x86_64 18/28 Verifying : javapackages-tools-3.4.1-11.el7.noarch 19/28 Verifying : drbd-xen-9.28.0-1.el7.x86_64 20/28 Verifying : nbdkit-1.8.0-4.el7.x86_64 21/28 Verifying : drbd-9.28.0-1.el7.x86_64 22/28 Verifying : drbd-udev-9.28.0-1.el7.x86_64 23/28 Verifying : drbd-bash-completion-9.28.0-1.el7.x86_64 24/28 Verifying : linstor-controller-1.29.0-1.el7_9.noarch 25/28 Verifying : drbd-reactor-1.4.1-1.x86_64 26/28 Verifying : xcp-ng-linstor-1.2-2.xcpng8.3.noarch 27/28 Verifying : alsa-lib-1.1.4.1-2.el7.x86_64 28/28 Installed: xcp-ng-linstor.noarch 0:1.2-2.xcpng8.3 Dependency Installed: alsa-lib.x86_64 0:1.1.4.1-2.el7 copy-jdk-configs.noarch 0:3.3-10.el7_5 drbd.x86_64 0:9.28.0-1.el7 drbd-bash-completion.x86_64 0:9.28.0-1.el7 drbd-pacemaker.x86_64 0:9.28.0-1.el7 drbd-reactor.x86_64 0:1.4.1-1 drbd-udev.x86_64 0:9.28.0-1.el7 drbd-utils.x86_64 0:9.28.0-1.el7 drbd-xen.x86_64 0:9.28.0-1.el7 graphite2.x86_64 0:1.3.10-1.el7_3 harfbuzz.x86_64 0:1.3.2-1.el7 http-nbd-transfer.x86_64 0:1.6.0-1.xcpng8.3 java-11-openjdk-headless.x86_64 1:11.0.23.0.9-2.el7_9 javapackages-tools.noarch 0:3.4.1-11.el7 kmod-drbd.x86_64 0:9.2.11-1.1.xcpng8.3 libxslt.x86_64 0:1.1.28-5.el7 linstor-client.noarch 0:1.23.0-1.xcpng8.3 linstor-common.noarch 0:1.29.0-1.el7_9 linstor-controller.noarch 0:1.29.0-1.el7_9 linstor-satellite.noarch 0:1.29.0-1.el7_9 lksctp-tools.x86_64 0:1.0.17-2.el7 nbdkit.x86_64 0:1.8.0-4.el7 pcsc-lite-libs.x86_64 0:1.8.8-8.el7 python-javapackages.noarch 0:3.4.1-11.el7 python-linstor.noarch 0:1.23.0-1.xcpng8.3 python-lxml.x86_64 0:3.2.1-4.el7 tzdata-java.noarch 0:2024a-1.el7 Complete! [00:59 node-3 ~]# xe-toolstack-restart Executing xe-toolstack-restart done. [00:59 node-3 ~]# xe pool-join master-address=XXXXX master-username=XXXX master-password="YYYYY" Host agent will restart and attempt to join pool in 10.000 seconds... -
Backup fail whereas xostor cluster is "healthy"
Hello,
My post title may be missleading.
I built a pool with 3 nodes and 3 replicas.
- node1
- node2
- node3
When i remove node 3 ( linstor forget + xo forget node ).
The snapshot of disks are not possible anymore.
SR_BACKEND_FAILURE_1200(, local variable 'e' referenced before assignment, )
However, i have 2 healthy node on 3. it should work no ?
️ I don't need support, it's just an observation in our test pool, maybe a bug, i report Best regards
-
Unable to add new node to pool using XOSTOR
Hello,
When you try to add a fresh node to a pool using XOSTOR
If you run the following error from CLI :
[00:54 node-3 ~]# xe pool-join master-address=XXXX master-username=XXX master-password="YYYY" The host joining the pool has an incompatible set of sm features from the pool coordinator. Make sure the sm are of the same versions and try again. pool_sm_ref: OpaqueRef:4d762998-79ee-8e88-1f0f-5b74aac9af85 candidate_sm_ref:OR this one thought XO
app.getLicenses is not a functionIt's maybe because you didn't installed Linstor package on it.
Run
yum install -y xcp-ng-release-linstor yum install -y xcp-ng-linstor xe-toolstack-restartOr simply use the install.sh script provided in the first post of this thread: https://xcp-ng.org/forum/topic/5361/xostor-hyperconvergence-preview
Then, you can add your node without issue
[00:59 node-3 ~]# xe pool-join master-address=XXXX master-username=XXXX master-password="YYYYY" Host agent will restart and attempt to join pool in 10.000 seconds... -
RE: Attempting to add new host fail on xoa and on server, worked on xcp-ng center
Hey @Danp ,
I can keep my cluster without this node up to Monday, do you have an idea / do you want to investigate on this case ? -
RE: Attempting to add new host fail on xoa and on server, worked on xcp-ng center
Hello, @Danp
I just reproduced it.
All my nodes are up to date
( my xo have 7 commits behind master, i read the commit's log of my missing commit, the commits are not related to my issue. ).In my scenario, i had a pool with 3 nodes
- node1
- node2
- node3
I reinstall node3 after a disaster ( i force forgot node 3).
Now, i can't add the host back to the pool.
I also tried to update the host after installation ( usually, i do this after ).
But it doesn't works more.
I will not add my node thought xcpng center to permit you further investigation
Here is the detailed operation json
{ "id": "0mcwehjzy", "properties": { "method": "pool.mergeInto", "params": { "sources": [ "64365465-fd4e-25b6-3db2-2cdcd9edba5e" ], "target": "a92ca4ca-caac-83b9-fa00-4bb75cb48f6c", "force": true }, "name": "API call: pool.mergeInto", "userId": "63a0dbaf-ba2d-4028-b80f-fe49f56686b2", "type": "api.call" }, "start": 1752092249470, "status": "failure", "updatedAt": 1752092249473, "end": 1752092249472, "result": { "message": "app.getLicenses is not a function", "name": "TypeError", "stack": "TypeError: app.getLicenses is not a function\n at enforceHostsHaveLicense (file:///etc/xen-orchestra/packages/xo-server/src/xo-mixins/pool.mjs:15:30)\n at Pools.apply (file:///etc/xen-orchestra/packages/xo-server/src/xo-mixins/pool.mjs:80:13)\n at Pools.mergeInto (/etc/xen-orchestra/node_modules/golike-defer/src/index.js:85:19)\n at Xo.mergeInto (file:///etc/xen-orchestra/packages/xo-server/src/api/pool.mjs:311:15)\n at Task.runInside (/etc/xen-orchestra/@vates/task/index.js:175:22)\n at Task.run (/etc/xen-orchestra/@vates/task/index.js:159:20)\n at Api.#callApiMethod (file:///etc/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:469:18)" } }``` -
RE: Recovery from lost node in HA
@olivierlambert No,
For once, i followed the installation step carefully ^^'
-
Recovery from lost node in HA
Hello,
I have a XCP-NG 8.3 pool running 3 hosts with XOSTOR in a 3 replicas with HA enabled.
This setup should permit to lose up to 2 nodes without dataloss
Initial informations:
- The linstor controller was on node 1.
- pool master was node 2
- Satellite are running in all nodes.
I was able to migrate VDI on XOSTOR successfuly ( even if when i start a transfert into xostor, i need to wait ~1 minute before the transfert really start ( i see that in XO ).
In my first tests, i will shut node 3 (which is neither master, not linstor controller )
For my first test, i didn't want to kill the linster controller host / pool master immediately, it should be my second test / third test )
I stopped node 3 ( poweroff from IPMI ).
However, then entire pool was dead.
In
xensource.logof all remaining nodes ( node 1, and node 2 ), i can see:Jul 5 15:32:20 node2 xapi: [debug||0 |Checking HA configuration D:9b97e277d80e|helpers] /usr/libexec/xapi/cluster-stack/xhad/ha_start_daemon exited with code 8 [stdout = ''; stderr = 'Sat Jul 5 15:32:20 CEST 2025 ha_start_daemon: the HA daemon stopped without forming a liveset (8)\x0A'] Jul 5 15:32:20 node2 xapi: [ warn||0 |Checking HA configuration D:9b97e277d80e|xapi_ha] /usr/libexec/xapi/cluster-stack/xhad/ha_start_daemon returned MTC_EXIT_CAN_NOT_ACCESS_STATEFILE (State-File is inaccessible) Jul 5 15:32:20 gco-002-rbx-002 xapi: [ warn||0 |Checking HA configuration D:9b97e277d80e|xapi_ha] ha_start_daemon failed with MTC_EXIT_CAN_NOT_ACCESS_STATEFILE: will contact existing master and check if HA is still enabledHowever, the storage layer was ok
[15:33 node1 linstor-controller]# linstor node list ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Node ┊ NodeType ┊ Addresses ┊ State ┊ ╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ h1 ┊ COMBINED ┊ 192.168.1.1:3366 (PLAIN) ┊ Online ┊ ┊ h2 ┊ COMBINED ┊ 192.168.1.2:3366 (PLAIN) ┊ Online ┊ ┊ h3 ┊ COMBINED ┊ 192.168.1.3:3366 (PLAIN) ┊ OFFLINE (Auto-eviction: 2025-07-05 16:33:42) ┊ ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯Volumes was also OK using
linstor volume list[15:33 r1 linstor-controller]# linstor volume list ╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ ┊ Node ┊ Resource ┊ StoragePool ┊ VolNr ┊ MinorNr ┊ DeviceName ┊ Allocated ┊ InUse ┊ State ┊ ╞═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡ ┊ r1 ┊ xcp-persistent-database ┊ xcp-sr-linstor_group_thin_device ┊ 0 ┊ 1000 ┊ /dev/drbd1000 ┊ 52.74 MiB ┊ InUse ┊ UpToDate ┊ ┊ r2 ┊ xcp-persistent-database ┊ xcp-sr-linstor_group_thin_device ┊ 0 ┊ 1000 ┊ /dev/drbd1000 ┊ 6.99 MiB ┊ Unused ┊ UpToDate ┊ ┊ r3 ┊ xcp-persistent-database ┊ xcp-sr-linstor_group_thin_device ┊ 0 ┊ 1000 ┊ /dev/drbd1000 ┊ 6.99 MiB ┊ ┊ Unknown ┊( i didn't put the entire list of volumes, in writing my post, i'm feel a bit stupid to don't save the entire output ).
I finally solved my issue by re-upping node3 which promote itself as master, but i need to perform this test again because the result is not the expected one.
Did i do something wrong ?
-
RE: High availability - host failure number
Hello, @nikade
Yeah i think like this already, but i'm surprised by the result of the calcul.
I should i have either: 3 nodes which can die, or "1", but i should not have the same number right ?
in my case, i run the command in a 3 hosts cluster and it return me
3as value. which disturb me.Best regards,
-
RE: High availability - host failure number
Hello @fred974
Did you find your answer ?
I don't understand clearly how a 4 nodes pool can tolerate 4 nodes dead.
-
RE: XOSTOR hyperconvergence preview
Hello,
I plan to install my XOSTOR cluster on a pool of 7 nodes with 3 replicas, but not all nodes at once because disks are in use.
consider:- node1
- node2
- node ...
- node 5
- node 6
- node 7.
with 2 disks on each
- sda: 128GB for the OS
- sdb: 1TB for local sr ( for now
)
I emptied node 6 & 7.
so, here is what i plan to do:
- On ALL NODES: setup linstor packages
Run the install script on node 6 & 7 to add their disks
so:node6# install.sh --disks /dev/sdb node7# install.sh --disks /dev/sdbThen, configure the SR and the linstor plugin manager as the following
xe sr-create \ type=linstor name-label=pool-01 \ host-uuid=XXXX \ device-config:group-name=linstor_group/thin_device device-config:redundancy=3 shared=true device-config:provisioning=thinNormally, i should have a linstor cluster running of 2 nodes ( 2 satellite and one controller randomly placed ) with only 2 disks and then, only 2/3 working replicas.
The cluster SHOULD be usable ( i'm right on this point ? )
The next step, would be to move VM from node 5 on it to evacuate node 5. and then add it to the cluster by the following
node5# install.sh --disks /dev/sdb node5# xe host-call-plugin \ host-uuid=node5-uuid \ plugin=linstor-manager \ fn=addHost args:groupName=linstor_group/thin_deviceThat should deploy satelite on node 5 and add the disk.
I normally should have 3/3 working replicas and can start to deploy others nodes progressively.
I'm right on the process ?
aS mentionned in the discord, i will post my feedbacks and results from my setup once i finalized it. ( maybe thought a blog post somewhere ).
Thanks to provide xostor in opensource, it's clearly the missing piece for this virtualization stack in opensource ( vs proxmox )
-
RE: XOSTOR on 8.3?
Hello, @TheNorthernLight
XOStor is now available in 8.3 since it's LTS