Offline
Hello, @DustinB
The https://vates.tech/xostor/ says:
The maximum size of any single Virtual Disk Image (VDI) will always be limited by the smallest disk in your cluster.
But in this case, maybe it can be stored in the "2TB disks" ? Maybe others can answer, i didn't test it.
hello @DustinB.
Yes you right, i would perform this to be able to have VDI with more than 1TB disk. ( which will not be possible because my smallest disk is 1TB (so, 879GB )...
This test permit to cover the following scenario:
Impact:
Expected results:
We didn't tests other filesystem than XFS for Linux based operating system because we use only XFS.
[hdevigne@VM1 ~]$ htop^C
[hdevigne@VM1 ~]$ echo "coucou" > test
-bash: test: Input/output error
[hdevigne@VM1 ~]$ dmesg
-bash: /usr/bin/dmesg: Input/output error
[hdevigne@VM1 ~]$ d^C
[hdevigne@VM1 ~]$ sudo -i
-bash: sudo: command not found
[hdevigne@VM1 ~]$ dm^C
[hdevigne@VM1 ~]$ sudo -i
-bash: sudo: command not found
[hdevigne@VM1 ~]$ dmesg
-bash: /usr/bin/dmesg: Input/output error
[hdevigne@VM1 ~]$ mount
-bash: mount: command not found
[hdevigne@VM1 ~]$ sud o-i
-bash: sud: command not found
[hdevigne@VM1 ~]$ sudo -i
As we predicted it, the vm is completly fucked-up
Windows VM crash and reboot in loop.
Linstor controller was on node 1, so we will not be able to see linstor nodes status, but we supposed they are in "disconnected" and in "pending eviction", but that doesn't matter a lot, disks are in read only, vm are fucked up after writing, it was our expected bevahior.
Re-plug node 1 and node 2.
Windows boot normally
Linux VM stays in a "broken state"
➜ ~ ssh VM1
suConnection closed by UNKNOWN port 65535
We didn't test a duration up to the eviction states of linstor nodes, but the documentation show that a linstor node restore would works ( see https://docs.xcp-ng.org/xostor/#what-to-do-when-a-node-is-in-an-evicted-state )
We didn't use HA at this time in the cluster, that could helped a bit in the recovery process. but in a precedent experience that i didn't "historize" like this one, the HA was completely down because it was not able to mount a file, i will probably write another topic on the forum to bring my results public.
Having HA change the criticity of the following note.
Thanks to @olivierlambert, @ronan and other people on the discord canal for answering to daily question which permit to this kind of tests to be made. As promissed, i put my result online 
Thanks for XOSTOR.
Futher tests to do: Retry with HA
Hello,
We tried the compression feature.
You "can see" a benefit only if you have a shared storage. ( and again, the migration between 2 nodes is already very fast, we don't see major difference, but maybe a VM will a lot of ram ( >32GB ) can see a difference.
If you don't have a shared storage ( like XOSTOR, NFS, ISCSI ), then you will not see any difference because there is a limitation of 30MB/s-40MB/s ( see here: https://xcp-ng.org/forum/topic/9389/backup-migration-performance )
Best regards,
Hello,
From my test, the result is having multiple xostor is not possible at this time. it's blocked.
( i didn't save the precise error message, but the error was clear: cannot have more than one XOSTOR in the pool ).
Hello, @burbilog
I think theses backup doesn't come from your backup plan.
Is there maybe snapshots on the VM ? 2022 is "very old"
Your full backups include memory which is not "classical" in a incremental backups.
Hello,
I tried on a new pool.
a little different scenario since i don't create xostor for now, on my previous example, i tried to add a node as replacement of an existing one..
I just run the install script only on node 1.
When i try make node2 join the pool, i reproduce the incompatible sm error i got previously.
The things which is "bizarre", is i don't have the license issue i got on Xen-orchestra. ( maybe it was finally not related ? )
Here is the complete logs.
pool.mergeInto
{
"sources": [
"17510fe0-db23-9414-f3df-2941bd34f8dc"
],
"target": "cc91fcdc-c7a8-a44c-65b3-a76dced49252",
"force": true
}
{
"code": "POOL_JOINING_SM_FEATURES_INCOMPATIBLE",
"params": [
"OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd",
""
],
"call": {
"duration": 1061,
"method": "pool.join_force",
"params": [
"* session id *",
"<MASTER_IP>",
"root",
"* obfuscated *"
]
},
"message": "POOL_JOINING_SM_FEATURES_INCOMPATIBLE(OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd, )",
"name": "XapiError",
"stack": "XapiError: POOL_JOINING_SM_FEATURES_INCOMPATIBLE(OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd, )
at Function.wrap (file:///etc/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12)
at file:///etc/xen-orchestra/packages/xen-api/transports/json-rpc.mjs:38:21
at runNextTicks (node:internal/process/task_queues:60:5)
at processImmediate (node:internal/timers:454:9)
at process.callbackTrampoline (node:internal/async_hooks:130:17)"
}```Hello,
we have nodes with multiple disk group.
Does someone experienced multiple xostor on different local group ?
In our case
It should works easily because we gonna register 2 xostor, but maybe should we put all disks in the same VG ? but in this case, the max vdi size will be lower ?
I purpose to edit the install script to provide group name selection to permit this use case easily.
It's ok ?
Hello @ronan-a
Just to be sure, you want logs of the node which want to join or the master ?
Bonne journée
Hello @ronan-a
I will reproduce the case, i will re-destroy one hypervisor and retrigger the case.
Thank you @ronan-a et @olivierlambert
If you need me to tests some special case don't hesit, we have a pool dedicated for this
Hello, @ronan-a
I will reinstall my hypervisor this week.
I will reproduce it and then, resend you the logs.
Bonne journée,
In this case @olivierlambert
The new node doesn't have any linstor version installed before.
I planned to install it AFTER adding it to a pool, but that's not possible. It's a "dependency" to join the pool. ( as i understand ^^ )
If my interpretation and solution is correct, can i propose a MR to add it thought the documentation ? ( for the CLI part at least, the xo side looks to be a bug for me )
Hello @olivierlambert
All my hosts were up to date ( verified many times ).
But once the linstor installed, the error disappeared.
Here is the my ssh i keeped open ( thanks Mac )
[00:55 node-3 ~]# xe pool-join master-address=XXXXX master-username=XXXX master-password="YYYYY"
The host joining the pool has an incompatible set of sm features from the pool coordinator. Make sure the sm are of the same versions and try again.
pool_sm_ref: OpaqueRef:4d762998-79ee-8e88-1f0f-5b74aac9af85
candidate_sm_ref:
[00:56 node-3 ~]# ^C
[00:58 node-3 ~]# yum install -y xcp-ng-release-linstor
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Excluding mirror: updates.xcp-ng.org
* xcp-ng-base: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
* xcp-ng-updates: mirrors.xcp-ng.org
Resolving Dependencies
--> Running transaction check
---> Package xcp-ng-release-linstor.noarch 0:1.4-2.xcpng8.3 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================================
Installing:
xcp-ng-release-linstor noarch 1.4-2.xcpng8.3 xcp-ng-updates 3.9 k
Transaction Summary
============================================================================================================================================================
Install 1 Package
Total download size: 3.9 k
Installed size: 477
Downloading packages:
xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch.rpm | 3.9 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch 1/1
Verifying : xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch 1/1
Installed:
xcp-ng-release-linstor.noarch 0:1.4-2.xcpng8.3
Complete!
[00:58 node-3 ~]# yum install -y xcp-ng-linstor
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
Excluding mirror: updates.xcp-ng.org
* xcp-ng-base: mirrors.xcp-ng.org
Excluding mirror: updates.xcp-ng.org
* xcp-ng-updates: mirrors.xcp-ng.org
xcp-ng-linstor/signature | 473 B 00:00:00
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-xcpng
Importing GPG key 0x3FD3AC9E:
Userid : "XCP-ng Key (XCP-ng Official Signing Key) <security@xcp-ng.org>"
Fingerprint: 34ac 2eb6 8248 fed6 d076 838a cd75 783a 3fd3 ac9e
Package : xcp-ng-release-8.3.0-32.x86_64 (@xcp:main/$releasever)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-xcpng
xcp-ng-linstor/signature | 3.0 kB 00:00:00 !!!
xcp-ng-linstor/primary_db | 26 kB 00:00:00
Resolving Dependencies
--> Running transaction check
---> Package xcp-ng-linstor.noarch 0:1.2-2.xcpng8.3 will be installed
--> Processing Dependency: drbd for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch
--> Processing Dependency: drbd-reactor for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch
--> Processing Dependency: http-nbd-transfer for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch
--> Processing Dependency: kmod-drbd for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch
--> Processing Dependency: linstor-client for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch
--> Processing Dependency: linstor-controller for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch
--> Processing Dependency: linstor-satellite for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch
--> Running transaction check
---> Package drbd.x86_64 0:9.28.0-1.el7 will be installed
--> Processing Dependency: drbd-bash-completion = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64
--> Processing Dependency: drbd-pacemaker = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64
--> Processing Dependency: drbd-udev = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64
--> Processing Dependency: drbd-utils = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64
--> Processing Dependency: drbd-xen = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64
---> Package drbd-reactor.x86_64 0:1.4.1-1 will be installed
---> Package http-nbd-transfer.x86_64 0:1.6.0-1.xcpng8.3 will be installed
--> Processing Dependency: nbdkit for package: http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64
---> Package kmod-drbd.x86_64 0:9.2.11-1.1.xcpng8.3 will be installed
---> Package linstor-client.noarch 0:1.23.0-1.xcpng8.3 will be installed
--> Processing Dependency: python-linstor >= 1.23.0 for package: linstor-client-1.23.0-1.xcpng8.3.noarch
---> Package linstor-controller.noarch 0:1.29.0-1.el7_9 will be installed
--> Processing Dependency: linstor-common = 1.29.0 for package: linstor-controller-1.29.0-1.el7_9.noarch
--> Processing Dependency: jre-11-headless for package: linstor-controller-1.29.0-1.el7_9.noarch
---> Package linstor-satellite.noarch 0:1.29.0-1.el7_9 will be installed
--> Running transaction check
---> Package drbd-bash-completion.x86_64 0:9.28.0-1.el7 will be installed
---> Package drbd-pacemaker.x86_64 0:9.28.0-1.el7 will be installed
---> Package drbd-udev.x86_64 0:9.28.0-1.el7 will be installed
---> Package drbd-utils.x86_64 0:9.28.0-1.el7 will be installed
---> Package drbd-xen.x86_64 0:9.28.0-1.el7 will be installed
---> Package java-11-openjdk-headless.x86_64 1:11.0.23.0.9-2.el7_9 will be installed
--> Processing Dependency: copy-jdk-configs >= 3.3 for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
--> Processing Dependency: tzdata-java >= 2023d for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
--> Processing Dependency: javapackages-tools for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
--> Processing Dependency: libasound.so.2(ALSA_0.9)(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
--> Processing Dependency: libasound.so.2(ALSA_0.9.0rc4)(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
--> Processing Dependency: lksctp-tools(x86-64) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
--> Processing Dependency: pcsc-lite-libs(x86-64) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
--> Processing Dependency: libasound.so.2()(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
--> Processing Dependency: libharfbuzz.so.0()(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64
---> Package linstor-common.noarch 0:1.29.0-1.el7_9 will be installed
---> Package nbdkit.x86_64 0:1.8.0-4.el7 will be installed
---> Package python-linstor.noarch 0:1.23.0-1.xcpng8.3 will be installed
--> Running transaction check
---> Package alsa-lib.x86_64 0:1.1.4.1-2.el7 will be installed
---> Package copy-jdk-configs.noarch 0:3.3-10.el7_5 will be installed
---> Package harfbuzz.x86_64 0:1.3.2-1.el7 will be installed
--> Processing Dependency: libgraphite2.so.3()(64bit) for package: harfbuzz-1.3.2-1.el7.x86_64
---> Package javapackages-tools.noarch 0:3.4.1-11.el7 will be installed
--> Processing Dependency: python-javapackages = 3.4.1-11.el7 for package: javapackages-tools-3.4.1-11.el7.noarch
--> Processing Dependency: libxslt for package: javapackages-tools-3.4.1-11.el7.noarch
---> Package lksctp-tools.x86_64 0:1.0.17-2.el7 will be installed
---> Package pcsc-lite-libs.x86_64 0:1.8.8-8.el7 will be installed
---> Package tzdata-java.noarch 0:2024a-1.el7 will be installed
--> Running transaction check
---> Package graphite2.x86_64 0:1.3.10-1.el7_3 will be installed
---> Package libxslt.x86_64 0:1.1.28-5.el7 will be installed
---> Package python-javapackages.noarch 0:3.4.1-11.el7 will be installed
--> Processing Dependency: python-lxml for package: python-javapackages-3.4.1-11.el7.noarch
--> Running transaction check
---> Package python-lxml.x86_64 0:3.2.1-4.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================================
Installing:
xcp-ng-linstor noarch 1.2-2.xcpng8.3 xcp-ng-linstor 3.6 k
Installing for dependencies:
alsa-lib x86_64 1.1.4.1-2.el7 xcp-ng-base 421 k
copy-jdk-configs noarch 3.3-10.el7_5 xcp-ng-base 21 k
drbd x86_64 9.28.0-1.el7 xcp-ng-linstor 22 k
drbd-bash-completion x86_64 9.28.0-1.el7 xcp-ng-linstor 8.5 k
drbd-pacemaker x86_64 9.28.0-1.el7 xcp-ng-linstor 47 k
drbd-reactor x86_64 1.4.1-1 xcp-ng-linstor 4.3 M
drbd-udev x86_64 9.28.0-1.el7 xcp-ng-linstor 7.0 k
drbd-utils x86_64 9.28.0-1.el7 xcp-ng-linstor 657 k
drbd-xen x86_64 9.28.0-1.el7 xcp-ng-linstor 9.8 k
graphite2 x86_64 1.3.10-1.el7_3 xcp-ng-base 115 k
harfbuzz x86_64 1.3.2-1.el7 xcp-ng-base 176 k
http-nbd-transfer x86_64 1.6.0-1.xcpng8.3 xcp-ng-updates 30 k
java-11-openjdk-headless x86_64 1:11.0.23.0.9-2.el7_9 xcp-ng-base 39 M
javapackages-tools noarch 3.4.1-11.el7 xcp-ng-base 72 k
kmod-drbd x86_64 9.2.11-1.1.xcpng8.3 xcp-ng-linstor 2.7 M
libxslt x86_64 1.1.28-5.el7 xcp-ng-base 241 k
linstor-client noarch 1.23.0-1.xcpng8.3 xcp-ng-linstor 266 k
linstor-common noarch 1.29.0-1.el7_9 xcp-ng-linstor 54 M
linstor-controller noarch 1.29.0-1.el7_9 xcp-ng-linstor 14 M
linstor-satellite noarch 1.29.0-1.el7_9 xcp-ng-linstor 880 k
lksctp-tools x86_64 1.0.17-2.el7 xcp-ng-base 87 k
nbdkit x86_64 1.8.0-4.el7 xcp-ng-base 57 k
pcsc-lite-libs x86_64 1.8.8-8.el7 xcp-ng-base 34 k
python-javapackages noarch 3.4.1-11.el7 xcp-ng-base 31 k
python-linstor noarch 1.23.0-1.xcpng8.3 xcp-ng-linstor 163 k
python-lxml x86_64 3.2.1-4.el7 xcp-ng-base 758 k
tzdata-java noarch 2024a-1.el7 xcp-ng-base 186 k
Transaction Summary
============================================================================================================================================================
Install 1 Package (+27 Dependent packages)
Total download size: 118 M
Installed size: 286 M
Downloading packages:
(1/28): drbd-9.28.0-1.el7.x86_64.rpm | 22 kB 00:00:00
(2/28): drbd-bash-completion-9.28.0-1.el7.x86_64.rpm | 8.5 kB 00:00:00
(3/28): drbd-pacemaker-9.28.0-1.el7.x86_64.rpm | 47 kB 00:00:00
(4/28): drbd-udev-9.28.0-1.el7.x86_64.rpm | 7.0 kB 00:00:00
(5/28): copy-jdk-configs-3.3-10.el7_5.noarch.rpm | 21 kB 00:00:00
(6/28): drbd-utils-9.28.0-1.el7.x86_64.rpm | 657 kB 00:00:00
(7/28): drbd-xen-9.28.0-1.el7.x86_64.rpm | 9.8 kB 00:00:00
(8/28): drbd-reactor-1.4.1-1.x86_64.rpm | 4.3 MB 00:00:00
(9/28): alsa-lib-1.1.4.1-2.el7.x86_64.rpm | 421 kB 00:00:00
(10/28): graphite2-1.3.10-1.el7_3.x86_64.rpm | 115 kB 00:00:00
(11/28): harfbuzz-1.3.2-1.el7.x86_64.rpm | 176 kB 00:00:00
(12/28): kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64.rpm | 2.7 MB 00:00:00
(13/28): javapackages-tools-3.4.1-11.el7.noarch.rpm | 72 kB 00:00:00
(14/28): linstor-client-1.23.0-1.xcpng8.3.noarch.rpm | 266 kB 00:00:00
(15/28): http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64.rpm | 30 kB 00:00:00
(16/28): libxslt-1.1.28-5.el7.x86_64.rpm | 241 kB 00:00:00
(17/28): linstor-controller-1.29.0-1.el7_9.noarch.rpm | 14 MB 00:00:00
(18/28): java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64.rpm | 39 MB 00:00:00
(19/28): linstor-satellite-1.29.0-1.el7_9.noarch.rpm | 880 kB 00:00:00
(20/28): lksctp-tools-1.0.17-2.el7.x86_64.rpm | 87 kB 00:00:00
(21/28): pcsc-lite-libs-1.8.8-8.el7.x86_64.rpm | 34 kB 00:00:00
(22/28): python-linstor-1.23.0-1.xcpng8.3.noarch.rpm | 163 kB 00:00:00
(23/28): nbdkit-1.8.0-4.el7.x86_64.rpm | 57 kB 00:00:00
(24/28): python-javapackages-3.4.1-11.el7.noarch.rpm | 31 kB 00:00:00
(25/28): xcp-ng-linstor-1.2-2.xcpng8.3.noarch.rpm | 3.6 kB 00:00:00
(26/28): python-lxml-3.2.1-4.el7.x86_64.rpm | 758 kB 00:00:00
(27/28): linstor-common-1.29.0-1.el7_9.noarch.rpm | 54 MB 00:00:00
(28/28): tzdata-java-2024a-1.el7.noarch.rpm | 186 kB 00:00:00
------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 88 MB/s | 118 MB 00:00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : drbd-utils-9.28.0-1.el7.x86_64 1/28
Installing : libxslt-1.1.28-5.el7.x86_64 2/28
Installing : tzdata-java-2024a-1.el7.noarch 3/28
Installing : python-lxml-3.2.1-4.el7.x86_64 4/28
Installing : python-javapackages-3.4.1-11.el7.noarch 5/28
Installing : javapackages-tools-3.4.1-11.el7.noarch 6/28
Installing : kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64 7/28
Installing : drbd-pacemaker-9.28.0-1.el7.x86_64 8/28
Installing : drbd-xen-9.28.0-1.el7.x86_64 9/28
Installing : drbd-udev-9.28.0-1.el7.x86_64 10/28
Installing : drbd-bash-completion-9.28.0-1.el7.x86_64 11/28
Installing : drbd-9.28.0-1.el7.x86_64 12/28
Installing : drbd-reactor-1.4.1-1.x86_64 13/28
Installing : alsa-lib-1.1.4.1-2.el7.x86_64 14/28
Installing : nbdkit-1.8.0-4.el7.x86_64 15/28
Installing : http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64 16/28
Installing : pcsc-lite-libs-1.8.8-8.el7.x86_64 17/28
Installing : lksctp-tools-1.0.17-2.el7.x86_64 18/28
Installing : python-linstor-1.23.0-1.xcpng8.3.noarch 19/28
Installing : linstor-client-1.23.0-1.xcpng8.3.noarch 20/28
Installing : copy-jdk-configs-3.3-10.el7_5.noarch 21/28
Installing : graphite2-1.3.10-1.el7_3.x86_64 22/28
Installing : harfbuzz-1.3.2-1.el7.x86_64 23/28
Installing : 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 24/28
Installing : linstor-common-1.29.0-1.el7_9.noarch 25/28
Installing : linstor-satellite-1.29.0-1.el7_9.noarch 26/28
Installing : linstor-controller-1.29.0-1.el7_9.noarch 27/28
Installing : xcp-ng-linstor-1.2-2.xcpng8.3.noarch 28/28
Verifying : drbd-utils-9.28.0-1.el7.x86_64 1/28
Verifying : graphite2-1.3.10-1.el7_3.x86_64 2/28
Verifying : python-lxml-3.2.1-4.el7.x86_64 3/28
Verifying : harfbuzz-1.3.2-1.el7.x86_64 4/28
Verifying : tzdata-java-2024a-1.el7.noarch 5/28
Verifying : copy-jdk-configs-3.3-10.el7_5.noarch 6/28
Verifying : python-javapackages-3.4.1-11.el7.noarch 7/28
Verifying : linstor-common-1.29.0-1.el7_9.noarch 8/28
Verifying : kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64 9/28
Verifying : drbd-pacemaker-9.28.0-1.el7.x86_64 10/28
Verifying : python-linstor-1.23.0-1.xcpng8.3.noarch 11/28
Verifying : 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 12/28
Verifying : lksctp-tools-1.0.17-2.el7.x86_64 13/28
Verifying : http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64 14/28
Verifying : linstor-client-1.23.0-1.xcpng8.3.noarch 15/28
Verifying : libxslt-1.1.28-5.el7.x86_64 16/28
Verifying : linstor-satellite-1.29.0-1.el7_9.noarch 17/28
Verifying : pcsc-lite-libs-1.8.8-8.el7.x86_64 18/28
Verifying : javapackages-tools-3.4.1-11.el7.noarch 19/28
Verifying : drbd-xen-9.28.0-1.el7.x86_64 20/28
Verifying : nbdkit-1.8.0-4.el7.x86_64 21/28
Verifying : drbd-9.28.0-1.el7.x86_64 22/28
Verifying : drbd-udev-9.28.0-1.el7.x86_64 23/28
Verifying : drbd-bash-completion-9.28.0-1.el7.x86_64 24/28
Verifying : linstor-controller-1.29.0-1.el7_9.noarch 25/28
Verifying : drbd-reactor-1.4.1-1.x86_64 26/28
Verifying : xcp-ng-linstor-1.2-2.xcpng8.3.noarch 27/28
Verifying : alsa-lib-1.1.4.1-2.el7.x86_64 28/28
Installed:
xcp-ng-linstor.noarch 0:1.2-2.xcpng8.3
Dependency Installed:
alsa-lib.x86_64 0:1.1.4.1-2.el7 copy-jdk-configs.noarch 0:3.3-10.el7_5 drbd.x86_64 0:9.28.0-1.el7
drbd-bash-completion.x86_64 0:9.28.0-1.el7 drbd-pacemaker.x86_64 0:9.28.0-1.el7 drbd-reactor.x86_64 0:1.4.1-1
drbd-udev.x86_64 0:9.28.0-1.el7 drbd-utils.x86_64 0:9.28.0-1.el7 drbd-xen.x86_64 0:9.28.0-1.el7
graphite2.x86_64 0:1.3.10-1.el7_3 harfbuzz.x86_64 0:1.3.2-1.el7 http-nbd-transfer.x86_64 0:1.6.0-1.xcpng8.3
java-11-openjdk-headless.x86_64 1:11.0.23.0.9-2.el7_9 javapackages-tools.noarch 0:3.4.1-11.el7 kmod-drbd.x86_64 0:9.2.11-1.1.xcpng8.3
libxslt.x86_64 0:1.1.28-5.el7 linstor-client.noarch 0:1.23.0-1.xcpng8.3 linstor-common.noarch 0:1.29.0-1.el7_9
linstor-controller.noarch 0:1.29.0-1.el7_9 linstor-satellite.noarch 0:1.29.0-1.el7_9 lksctp-tools.x86_64 0:1.0.17-2.el7
nbdkit.x86_64 0:1.8.0-4.el7 pcsc-lite-libs.x86_64 0:1.8.8-8.el7 python-javapackages.noarch 0:3.4.1-11.el7
python-linstor.noarch 0:1.23.0-1.xcpng8.3 python-lxml.x86_64 0:3.2.1-4.el7 tzdata-java.noarch 0:2024a-1.el7
Complete!
[00:59 node-3 ~]# xe-toolstack-restart
Executing xe-toolstack-restart
done.
[00:59 node-3 ~]# xe pool-join master-address=XXXXX master-username=XXXX master-password="YYYYY"
Host agent will restart and attempt to join pool in 10.000 seconds...
Hello,
My post title may be missleading.
I built a pool with 3 nodes and 3 replicas.
When i remove node 3 ( linstor forget + xo forget node ).
The snapshot of disks are not possible anymore.
SR_BACKEND_FAILURE_1200(, local variable 'e' referenced before assignment, )
However, i have 2 healthy node on 3. it should work no ?
️ I don't need support, it's just an observation in our test pool, maybe a bug, i report
Best regards
Hello,
When you try to add a fresh node to a pool using XOSTOR
If you run the following error from CLI :
[00:54 node-3 ~]# xe pool-join master-address=XXXX master-username=XXX master-password="YYYY"
The host joining the pool has an incompatible set of sm features from the pool coordinator. Make sure the sm are of the same versions and try again.
pool_sm_ref: OpaqueRef:4d762998-79ee-8e88-1f0f-5b74aac9af85
candidate_sm_ref:
OR this one thought XO
app.getLicenses is not a function
It's maybe because you didn't installed Linstor package on it.
Run
yum install -y xcp-ng-release-linstor
yum install -y xcp-ng-linstor
xe-toolstack-restart
Or simply use the install.sh script provided in the first post of this thread: https://xcp-ng.org/forum/topic/5361/xostor-hyperconvergence-preview
Then, you can add your node without issue
[00:59 node-3 ~]# xe pool-join master-address=XXXX master-username=XXXX master-password="YYYYY"
Host agent will restart and attempt to join pool in 10.000 seconds...
Hey @Danp ,
I can keep my cluster without this node up to Monday, do you have an idea / do you want to investigate on this case ?
Hello, @Danp
I just reproduced it.
All my nodes are up to date
( my xo have 7 commits behind master, i read the commit's log of my missing commit, the commits are not related to my issue. ).
In my scenario, i had a pool with 3 nodes
I reinstall node3 after a disaster ( i force forgot node 3).
Now, i can't add the host back to the pool.
I also tried to update the host after installation ( usually, i do this after ).
But it doesn't works more.
I will not add my node thought xcpng center to permit you further investigation
Here is the detailed operation json
{
"id": "0mcwehjzy",
"properties": {
"method": "pool.mergeInto",
"params": {
"sources": [
"64365465-fd4e-25b6-3db2-2cdcd9edba5e"
],
"target": "a92ca4ca-caac-83b9-fa00-4bb75cb48f6c",
"force": true
},
"name": "API call: pool.mergeInto",
"userId": "63a0dbaf-ba2d-4028-b80f-fe49f56686b2",
"type": "api.call"
},
"start": 1752092249470,
"status": "failure",
"updatedAt": 1752092249473,
"end": 1752092249472,
"result": {
"message": "app.getLicenses is not a function",
"name": "TypeError",
"stack": "TypeError: app.getLicenses is not a function\n at enforceHostsHaveLicense (file:///etc/xen-orchestra/packages/xo-server/src/xo-mixins/pool.mjs:15:30)\n at Pools.apply (file:///etc/xen-orchestra/packages/xo-server/src/xo-mixins/pool.mjs:80:13)\n at Pools.mergeInto (/etc/xen-orchestra/node_modules/golike-defer/src/index.js:85:19)\n at Xo.mergeInto (file:///etc/xen-orchestra/packages/xo-server/src/api/pool.mjs:311:15)\n at Task.runInside (/etc/xen-orchestra/@vates/task/index.js:175:22)\n at Task.run (/etc/xen-orchestra/@vates/task/index.js:159:20)\n at Api.#callApiMethod (file:///etc/xen-orchestra/packages/xo-server/src/xo-mixins/api.mjs:469:18)"
}
}```@olivierlambert No,
For once, i followed the installation step carefully ^^'