Hello @florent ,
You mean, "check" right ? because if it's doesn't check, the job is finished before the merge and the merge run in background in xen orchestra.
Best regards,
Offline
Posts
-
RE: Lock file is already being held whereas no backup are running.
-
RE: Lock file is already being held whereas no backup are running.
Hello, @Pilow
Thanks for the tips !
But it would be great to see it in the Task UI to find more easily the trick.
-
Lock file is already being held whereas no backup are running.
Hello,
Sometimes, i have on some jobs ( which run every 4 hours ).
Howerver, the previous job is finished in success
Is it possible that has a relation with the mergeWorker of backups which could be running ? if it doesn't finished his operations ?
Example of logs:
xen-orchestra | 2025-11-27T00:44:43.487Z xo:backups:mergeWorker INFO merge in progress { xen-orchestra | done: 2057, xen-orchestra | parent: '/xo-vm-backups/f37e259d-beaa-7617-e6f1-be814f21e056/vdis/29e0185e-2f67-44d4-bb9e-ee2a772e2543/b09c0230-219f-4ddf-8e19-bfed1464014f/20251126T070610Z.vhd', xen-orchestra | progress: 25, xen-orchestra | total: 8128 xen-orchestra | }Is it possible to have this in the XO Tasks sections ? it's interessant to see this.
Best regards
-
RE: Native Ceph RBD SM driver for XCP-ng
Hello,
Thanks for your work !
We have some hypervisors of tests at Gladhost, we can use them with pleasure to test your work on xcp-ng 8.3 !
Best regards
-
RE: XOSTOR hyperconvergence preview
Hello,
I got my whole xostor destroyed, i don't know how precisely.
I found some errors in sattelite
Error context: An error occurred while processing resource 'Node: 'host', Rsc: 'xcp-volume-e011c043-8751-45e6-be06-4ce9f8807cad'' ErrorContext: Details: Command 'lvcreate --config 'devices { filter=['"'"'a|/dev/md127|'"'"','"'"'a|/dev/md126p3|'"'"','"'"'r|.*|'"'"'] }' --virtualsize 52543488k linstor_primary --thinpool thin_device --name xcp-volume-e011c043-8751-45e6-be06-4ce9f8807cad_00000' returned with exitcode 5. Standard out: Error message: WARNING: Remaining free space in metadata of thin pool linstor_primary/thin_device is too low (98.06% >= 96.30%). Resize is recommended. Cannot create new thin volume, free space in thin pool linstor_primary/thin_device reached threshold.of course, i checked, my SR was not full
And the controller crashed, and i couldn't make it works.
Here is the error i got
========== Category: RuntimeException Class name: IllegalStateException Class canonical name: java.lang.IllegalStateException Generated at: Method 'newIllegalStateException', Source file 'DataUtils.java', Line #870 Error message: Reading from nio:/var/lib/linstor/linstordb.mv.db failed; file length 2293760 read length 384 at 2445540 [1.4.197/1]So i deduce the database was fucked-up, i tried to open the file as explained in the documentation, but the linstor schema was "not found" in the file, event if using
cati see data about it.for now, i leave xostor and i'm back to localstorage until we know what to do when this issue occured with a "solution path".
-
RE: Multiple disks groups
Hello, @DustinB
The https://vates.tech/xostor/ says:
The maximum size of any single Virtual Disk Image (VDI) will always be limited by the smallest disk in your cluster.
But in this case, maybe it can be stored in the "2TB disks" ? Maybe others can answer, i didn't test it.
-
RE: Multiple disks groups
hello @DustinB.
Yes you right, i would perform this to be able to have VDI with more than 1TB disk. ( which will not be possible because my smallest disk is 1TB (so, 879GB )...
-
XOSTOR Global network disruption test
Summary
This test permit to cover the following scenario:
- Storage network is down
- All networks are down
Impact:
- All hosts cannot see each others
- Linstor DRDB replication can no longer works ( everyone will be in readonly ).
Expected results:
- All write operation on VM fails.
- a reboot solve the issue.
Environments
- 3 hypervisors
- Node 1
- Node 2
- Node 3 ( Master )
- 13 vms ( mixed of Windows Server and Rocky Linux VMs on XFS, No kubernetes in this section ).
- VM observation point is : VM1
We didn't tests other filesystem than XFS for Linux based operating system because we use only XFS.
Procedure
- Unplug network cable from all non-master node:
- Node 1
- Node 2
- Keep networks only on the master xcpng node to keep management and observe the behavior
- Access to a VM located in the master node ( which is still reacheable ).
- Try to write on VM1, ensure that you have "I/O" error.
- Wait 5 minutes
- Re-add back node 1 & Node 2
- Check states of all VMs.
- Reboot them if needed.
Execution
- Cable disconnected from node 1 and node 2
- From VM1, we have
[hdevigne@VM1 ~]$ htop^C [hdevigne@VM1 ~]$ echo "coucou" > test -bash: test: Input/output error [hdevigne@VM1 ~]$ dmesg -bash: /usr/bin/dmesg: Input/output error [hdevigne@VM1 ~]$ d^C [hdevigne@VM1 ~]$ sudo -i -bash: sudo: command not found [hdevigne@VM1 ~]$ dm^C [hdevigne@VM1 ~]$ sudo -i -bash: sudo: command not found [hdevigne@VM1 ~]$ dmesg -bash: /usr/bin/dmesg: Input/output error [hdevigne@VM1 ~]$ mount -bash: mount: command not found [hdevigne@VM1 ~]$ sud o-i -bash: sud: command not found [hdevigne@VM1 ~]$ sudo -i
As we predicted it, the vm is completly fucked-up 
-
Windows VM crash and reboot in loop.
-
Linstor controller was on node 1, so we will not be able to see linstor nodes status, but we supposed they are in "disconnected" and in "pending eviction", but that doesn't matter a lot, disks are in read only, vm are fucked up after writing, it was our expected bevahior.
-
Re-plug node 1 and node 2.
-
Windows boot normally
-
Linux VM stays in a "broken state"
➜ ~ ssh VM1 suConnection closed by UNKNOWN port 65535- Force rebooting all VMs from Xen-orchestra permit to revert all vms to a correct state
Limitation of the test
We didn't test a duration up to the eviction states of linstor nodes, but the documentation show that a
linstor node restorewould works ( see https://docs.xcp-ng.org/xostor/#what-to-do-when-a-node-is-in-an-evicted-state )
We didn't use HA at this time in the cluster, that could helped a bit in the recovery process. but in a precedent experience that i didn't "historize" like this one, the HA was completely down because it was not able to mount a file, i will probably write another topic on the forum to bring my results public.Important notes
Having HA change the criticity of the following note.
- This test show us that while we don't have HA, all Management components should NOT be placed in XOSTOR to avoid loosing access to it uppon reboot of the VM.
- If we maintain the idea to put Management component ( XO, Firewall etc... ) in the xostor without HA, we aim to increase recovery time because the recovery will be "manual" from IPMI.
- Maybe should we simply force reboot nodes after network recovery ? but a bit violent, HA works like this.
Credit
Thanks to @olivierlambert, @ronan and other people on the discord canal for answering to daily question which permit to this kind of tests to be made. As promissed, i put my result online
Thanks for XOSTOR.
Futher tests to do: Retry with HA
-
RE: Migration compression is not available on this pool
Hello,
We tried the compression feature.
You "can see" a benefit only if you have a shared storage. ( and again, the migration between 2 nodes is already very fast, we don't see major difference, but maybe a VM will a lot of ram ( >32GB ) can see a difference.
If you don't have a shared storage ( like XOSTOR, NFS, ISCSI ), then you will not see any difference because there is a limitation of 30MB/s-40MB/s ( see here: https://xcp-ng.org/forum/topic/9389/backup-migration-performance )
Best regards,
-
RE: Multiple disks groups
Hello,
From my test, the result is having multiple xostor is not possible at this time. it's blocked.
( i didn't save the precise error message, but the error was clear: cannot have more than one XOSTOR in the pool ).
-
RE: Full or not?
Hello, @burbilog
I think theses backup doesn't come from your backup plan.
Is there maybe snapshots on the VM ? 2022 is "very old"
Your full backups include memory which is not "classical" in a incremental backups.
-
RE: Unable to add new node to pool using XOSTOR
Hello,
I tried on a new pool.
a little different scenario since i don't create xostor for now, on my previous example, i tried to add a node as replacement of an existing one..
I just run the install script only on node 1.
When i try make node2 join the pool, i reproduce the incompatible sm error i got previously.
The things which is "bizarre", is i don't have the license issue i got on Xen-orchestra. ( maybe it was finally not related ? )
Here is the complete logs.
pool.mergeInto { "sources": [ "17510fe0-db23-9414-f3df-2941bd34f8dc" ], "target": "cc91fcdc-c7a8-a44c-65b3-a76dced49252", "force": true } { "code": "POOL_JOINING_SM_FEATURES_INCOMPATIBLE", "params": [ "OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd", "" ], "call": { "duration": 1061, "method": "pool.join_force", "params": [ "* session id *", "<MASTER_IP>", "root", "* obfuscated *" ] }, "message": "POOL_JOINING_SM_FEATURES_INCOMPATIBLE(OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd, )", "name": "XapiError", "stack": "XapiError: POOL_JOINING_SM_FEATURES_INCOMPATIBLE(OpaqueRef:090b8da1-9654-066c-84f9-7ab15cb101fd, ) at Function.wrap (file:///etc/xen-orchestra/packages/xen-api/_XapiError.mjs:16:12) at file:///etc/xen-orchestra/packages/xen-api/transports/json-rpc.mjs:38:21 at runNextTicks (node:internal/process/task_queues:60:5) at processImmediate (node:internal/timers:454:9) at process.callbackTrampoline (node:internal/async_hooks:130:17)" }``` -
Multiple disks groups
Hello,
we have nodes with multiple disk group.
Does someone experienced multiple xostor on different local group ?
In our case
- 2x1TB & 2x2TB
It should works easily because we gonna register 2 xostor, but maybe should we put all disks in the same VG ? but in this case, the max vdi size will be lower ?
I purpose to edit the install script to provide group name selection to permit this use case easily.
It's ok ?
-
RE: Unable to add new node to pool using XOSTOR
Hello @ronan-a
Just to be sure, you want logs of the node which want to join or the master ?
Bonne journée
-
RE: Backup fail whereas xostor cluster is "healthy"
Hello @ronan-a
I will reproduce the case, i will re-destroy one hypervisor and retrigger the case.
Thank you @ronan-a et @olivierlambert
If you need me to tests some special case don't hesit, we have a pool dedicated for this
-
RE: Unable to add new node to pool using XOSTOR
Hello, @ronan-a
I will reinstall my hypervisor this week.
I will reproduce it and then, resend you the logs.
Bonne journée,
-
RE: Unable to add new node to pool using XOSTOR
In this case @olivierlambert
The new node doesn't have any linstor version installed before.
I planned to install it AFTER adding it to a pool, but that's not possible. It's a "dependency" to join the pool. ( as i understand ^^ )
If my interpretation and solution is correct, can i propose a MR to add it thought the documentation ? ( for the CLI part at least, the xo side looks to be a bug for me )
-
RE: Unable to add new node to pool using XOSTOR
Hello @olivierlambert
All my hosts were up to date ( verified many times ).
But once the linstor installed, the error disappeared.
Here is the my ssh i keeped open
( thanks Mac )[00:55 node-3 ~]# xe pool-join master-address=XXXXX master-username=XXXX master-password="YYYYY" The host joining the pool has an incompatible set of sm features from the pool coordinator. Make sure the sm are of the same versions and try again. pool_sm_ref: OpaqueRef:4d762998-79ee-8e88-1f0f-5b74aac9af85 candidate_sm_ref: [00:56 node-3 ~]# ^C [00:58 node-3 ~]# yum install -y xcp-ng-release-linstor Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Excluding mirror: updates.xcp-ng.org * xcp-ng-base: mirrors.xcp-ng.org Excluding mirror: updates.xcp-ng.org * xcp-ng-updates: mirrors.xcp-ng.org Resolving Dependencies --> Running transaction check ---> Package xcp-ng-release-linstor.noarch 0:1.4-2.xcpng8.3 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================ Package Arch Version Repository Size ============================================================================================================================================================ Installing: xcp-ng-release-linstor noarch 1.4-2.xcpng8.3 xcp-ng-updates 3.9 k Transaction Summary ============================================================================================================================================================ Install 1 Package Total download size: 3.9 k Installed size: 477 Downloading packages: xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch.rpm | 3.9 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch 1/1 Verifying : xcp-ng-release-linstor-1.4-2.xcpng8.3.noarch 1/1 Installed: xcp-ng-release-linstor.noarch 0:1.4-2.xcpng8.3 Complete! [00:58 node-3 ~]# yum install -y xcp-ng-linstor Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Excluding mirror: updates.xcp-ng.org * xcp-ng-base: mirrors.xcp-ng.org Excluding mirror: updates.xcp-ng.org * xcp-ng-updates: mirrors.xcp-ng.org xcp-ng-linstor/signature | 473 B 00:00:00 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-xcpng Importing GPG key 0x3FD3AC9E: Userid : "XCP-ng Key (XCP-ng Official Signing Key) <security@xcp-ng.org>" Fingerprint: 34ac 2eb6 8248 fed6 d076 838a cd75 783a 3fd3 ac9e Package : xcp-ng-release-8.3.0-32.x86_64 (@xcp:main/$releasever) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-xcpng xcp-ng-linstor/signature | 3.0 kB 00:00:00 !!! xcp-ng-linstor/primary_db | 26 kB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package xcp-ng-linstor.noarch 0:1.2-2.xcpng8.3 will be installed --> Processing Dependency: drbd for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: drbd-reactor for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: http-nbd-transfer for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: kmod-drbd for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: linstor-client for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: linstor-controller for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Processing Dependency: linstor-satellite for package: xcp-ng-linstor-1.2-2.xcpng8.3.noarch --> Running transaction check ---> Package drbd.x86_64 0:9.28.0-1.el7 will be installed --> Processing Dependency: drbd-bash-completion = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 --> Processing Dependency: drbd-pacemaker = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 --> Processing Dependency: drbd-udev = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 --> Processing Dependency: drbd-utils = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 --> Processing Dependency: drbd-xen = 9.28.0-1.el7 for package: drbd-9.28.0-1.el7.x86_64 ---> Package drbd-reactor.x86_64 0:1.4.1-1 will be installed ---> Package http-nbd-transfer.x86_64 0:1.6.0-1.xcpng8.3 will be installed --> Processing Dependency: nbdkit for package: http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64 ---> Package kmod-drbd.x86_64 0:9.2.11-1.1.xcpng8.3 will be installed ---> Package linstor-client.noarch 0:1.23.0-1.xcpng8.3 will be installed --> Processing Dependency: python-linstor >= 1.23.0 for package: linstor-client-1.23.0-1.xcpng8.3.noarch ---> Package linstor-controller.noarch 0:1.29.0-1.el7_9 will be installed --> Processing Dependency: linstor-common = 1.29.0 for package: linstor-controller-1.29.0-1.el7_9.noarch --> Processing Dependency: jre-11-headless for package: linstor-controller-1.29.0-1.el7_9.noarch ---> Package linstor-satellite.noarch 0:1.29.0-1.el7_9 will be installed --> Running transaction check ---> Package drbd-bash-completion.x86_64 0:9.28.0-1.el7 will be installed ---> Package drbd-pacemaker.x86_64 0:9.28.0-1.el7 will be installed ---> Package drbd-udev.x86_64 0:9.28.0-1.el7 will be installed ---> Package drbd-utils.x86_64 0:9.28.0-1.el7 will be installed ---> Package drbd-xen.x86_64 0:9.28.0-1.el7 will be installed ---> Package java-11-openjdk-headless.x86_64 1:11.0.23.0.9-2.el7_9 will be installed --> Processing Dependency: copy-jdk-configs >= 3.3 for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: tzdata-java >= 2023d for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: javapackages-tools for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: libasound.so.2(ALSA_0.9)(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: libasound.so.2(ALSA_0.9.0rc4)(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: lksctp-tools(x86-64) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: pcsc-lite-libs(x86-64) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: libasound.so.2()(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 --> Processing Dependency: libharfbuzz.so.0()(64bit) for package: 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 ---> Package linstor-common.noarch 0:1.29.0-1.el7_9 will be installed ---> Package nbdkit.x86_64 0:1.8.0-4.el7 will be installed ---> Package python-linstor.noarch 0:1.23.0-1.xcpng8.3 will be installed --> Running transaction check ---> Package alsa-lib.x86_64 0:1.1.4.1-2.el7 will be installed ---> Package copy-jdk-configs.noarch 0:3.3-10.el7_5 will be installed ---> Package harfbuzz.x86_64 0:1.3.2-1.el7 will be installed --> Processing Dependency: libgraphite2.so.3()(64bit) for package: harfbuzz-1.3.2-1.el7.x86_64 ---> Package javapackages-tools.noarch 0:3.4.1-11.el7 will be installed --> Processing Dependency: python-javapackages = 3.4.1-11.el7 for package: javapackages-tools-3.4.1-11.el7.noarch --> Processing Dependency: libxslt for package: javapackages-tools-3.4.1-11.el7.noarch ---> Package lksctp-tools.x86_64 0:1.0.17-2.el7 will be installed ---> Package pcsc-lite-libs.x86_64 0:1.8.8-8.el7 will be installed ---> Package tzdata-java.noarch 0:2024a-1.el7 will be installed --> Running transaction check ---> Package graphite2.x86_64 0:1.3.10-1.el7_3 will be installed ---> Package libxslt.x86_64 0:1.1.28-5.el7 will be installed ---> Package python-javapackages.noarch 0:3.4.1-11.el7 will be installed --> Processing Dependency: python-lxml for package: python-javapackages-3.4.1-11.el7.noarch --> Running transaction check ---> Package python-lxml.x86_64 0:3.2.1-4.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================ Package Arch Version Repository Size ============================================================================================================================================================ Installing: xcp-ng-linstor noarch 1.2-2.xcpng8.3 xcp-ng-linstor 3.6 k Installing for dependencies: alsa-lib x86_64 1.1.4.1-2.el7 xcp-ng-base 421 k copy-jdk-configs noarch 3.3-10.el7_5 xcp-ng-base 21 k drbd x86_64 9.28.0-1.el7 xcp-ng-linstor 22 k drbd-bash-completion x86_64 9.28.0-1.el7 xcp-ng-linstor 8.5 k drbd-pacemaker x86_64 9.28.0-1.el7 xcp-ng-linstor 47 k drbd-reactor x86_64 1.4.1-1 xcp-ng-linstor 4.3 M drbd-udev x86_64 9.28.0-1.el7 xcp-ng-linstor 7.0 k drbd-utils x86_64 9.28.0-1.el7 xcp-ng-linstor 657 k drbd-xen x86_64 9.28.0-1.el7 xcp-ng-linstor 9.8 k graphite2 x86_64 1.3.10-1.el7_3 xcp-ng-base 115 k harfbuzz x86_64 1.3.2-1.el7 xcp-ng-base 176 k http-nbd-transfer x86_64 1.6.0-1.xcpng8.3 xcp-ng-updates 30 k java-11-openjdk-headless x86_64 1:11.0.23.0.9-2.el7_9 xcp-ng-base 39 M javapackages-tools noarch 3.4.1-11.el7 xcp-ng-base 72 k kmod-drbd x86_64 9.2.11-1.1.xcpng8.3 xcp-ng-linstor 2.7 M libxslt x86_64 1.1.28-5.el7 xcp-ng-base 241 k linstor-client noarch 1.23.0-1.xcpng8.3 xcp-ng-linstor 266 k linstor-common noarch 1.29.0-1.el7_9 xcp-ng-linstor 54 M linstor-controller noarch 1.29.0-1.el7_9 xcp-ng-linstor 14 M linstor-satellite noarch 1.29.0-1.el7_9 xcp-ng-linstor 880 k lksctp-tools x86_64 1.0.17-2.el7 xcp-ng-base 87 k nbdkit x86_64 1.8.0-4.el7 xcp-ng-base 57 k pcsc-lite-libs x86_64 1.8.8-8.el7 xcp-ng-base 34 k python-javapackages noarch 3.4.1-11.el7 xcp-ng-base 31 k python-linstor noarch 1.23.0-1.xcpng8.3 xcp-ng-linstor 163 k python-lxml x86_64 3.2.1-4.el7 xcp-ng-base 758 k tzdata-java noarch 2024a-1.el7 xcp-ng-base 186 k Transaction Summary ============================================================================================================================================================ Install 1 Package (+27 Dependent packages) Total download size: 118 M Installed size: 286 M Downloading packages: (1/28): drbd-9.28.0-1.el7.x86_64.rpm | 22 kB 00:00:00 (2/28): drbd-bash-completion-9.28.0-1.el7.x86_64.rpm | 8.5 kB 00:00:00 (3/28): drbd-pacemaker-9.28.0-1.el7.x86_64.rpm | 47 kB 00:00:00 (4/28): drbd-udev-9.28.0-1.el7.x86_64.rpm | 7.0 kB 00:00:00 (5/28): copy-jdk-configs-3.3-10.el7_5.noarch.rpm | 21 kB 00:00:00 (6/28): drbd-utils-9.28.0-1.el7.x86_64.rpm | 657 kB 00:00:00 (7/28): drbd-xen-9.28.0-1.el7.x86_64.rpm | 9.8 kB 00:00:00 (8/28): drbd-reactor-1.4.1-1.x86_64.rpm | 4.3 MB 00:00:00 (9/28): alsa-lib-1.1.4.1-2.el7.x86_64.rpm | 421 kB 00:00:00 (10/28): graphite2-1.3.10-1.el7_3.x86_64.rpm | 115 kB 00:00:00 (11/28): harfbuzz-1.3.2-1.el7.x86_64.rpm | 176 kB 00:00:00 (12/28): kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64.rpm | 2.7 MB 00:00:00 (13/28): javapackages-tools-3.4.1-11.el7.noarch.rpm | 72 kB 00:00:00 (14/28): linstor-client-1.23.0-1.xcpng8.3.noarch.rpm | 266 kB 00:00:00 (15/28): http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64.rpm | 30 kB 00:00:00 (16/28): libxslt-1.1.28-5.el7.x86_64.rpm | 241 kB 00:00:00 (17/28): linstor-controller-1.29.0-1.el7_9.noarch.rpm | 14 MB 00:00:00 (18/28): java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64.rpm | 39 MB 00:00:00 (19/28): linstor-satellite-1.29.0-1.el7_9.noarch.rpm | 880 kB 00:00:00 (20/28): lksctp-tools-1.0.17-2.el7.x86_64.rpm | 87 kB 00:00:00 (21/28): pcsc-lite-libs-1.8.8-8.el7.x86_64.rpm | 34 kB 00:00:00 (22/28): python-linstor-1.23.0-1.xcpng8.3.noarch.rpm | 163 kB 00:00:00 (23/28): nbdkit-1.8.0-4.el7.x86_64.rpm | 57 kB 00:00:00 (24/28): python-javapackages-3.4.1-11.el7.noarch.rpm | 31 kB 00:00:00 (25/28): xcp-ng-linstor-1.2-2.xcpng8.3.noarch.rpm | 3.6 kB 00:00:00 (26/28): python-lxml-3.2.1-4.el7.x86_64.rpm | 758 kB 00:00:00 (27/28): linstor-common-1.29.0-1.el7_9.noarch.rpm | 54 MB 00:00:00 (28/28): tzdata-java-2024a-1.el7.noarch.rpm | 186 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------ Total 88 MB/s | 118 MB 00:00:01 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : drbd-utils-9.28.0-1.el7.x86_64 1/28 Installing : libxslt-1.1.28-5.el7.x86_64 2/28 Installing : tzdata-java-2024a-1.el7.noarch 3/28 Installing : python-lxml-3.2.1-4.el7.x86_64 4/28 Installing : python-javapackages-3.4.1-11.el7.noarch 5/28 Installing : javapackages-tools-3.4.1-11.el7.noarch 6/28 Installing : kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64 7/28 Installing : drbd-pacemaker-9.28.0-1.el7.x86_64 8/28 Installing : drbd-xen-9.28.0-1.el7.x86_64 9/28 Installing : drbd-udev-9.28.0-1.el7.x86_64 10/28 Installing : drbd-bash-completion-9.28.0-1.el7.x86_64 11/28 Installing : drbd-9.28.0-1.el7.x86_64 12/28 Installing : drbd-reactor-1.4.1-1.x86_64 13/28 Installing : alsa-lib-1.1.4.1-2.el7.x86_64 14/28 Installing : nbdkit-1.8.0-4.el7.x86_64 15/28 Installing : http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64 16/28 Installing : pcsc-lite-libs-1.8.8-8.el7.x86_64 17/28 Installing : lksctp-tools-1.0.17-2.el7.x86_64 18/28 Installing : python-linstor-1.23.0-1.xcpng8.3.noarch 19/28 Installing : linstor-client-1.23.0-1.xcpng8.3.noarch 20/28 Installing : copy-jdk-configs-3.3-10.el7_5.noarch 21/28 Installing : graphite2-1.3.10-1.el7_3.x86_64 22/28 Installing : harfbuzz-1.3.2-1.el7.x86_64 23/28 Installing : 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 24/28 Installing : linstor-common-1.29.0-1.el7_9.noarch 25/28 Installing : linstor-satellite-1.29.0-1.el7_9.noarch 26/28 Installing : linstor-controller-1.29.0-1.el7_9.noarch 27/28 Installing : xcp-ng-linstor-1.2-2.xcpng8.3.noarch 28/28 Verifying : drbd-utils-9.28.0-1.el7.x86_64 1/28 Verifying : graphite2-1.3.10-1.el7_3.x86_64 2/28 Verifying : python-lxml-3.2.1-4.el7.x86_64 3/28 Verifying : harfbuzz-1.3.2-1.el7.x86_64 4/28 Verifying : tzdata-java-2024a-1.el7.noarch 5/28 Verifying : copy-jdk-configs-3.3-10.el7_5.noarch 6/28 Verifying : python-javapackages-3.4.1-11.el7.noarch 7/28 Verifying : linstor-common-1.29.0-1.el7_9.noarch 8/28 Verifying : kmod-drbd-9.2.11-1.1.xcpng8.3.x86_64 9/28 Verifying : drbd-pacemaker-9.28.0-1.el7.x86_64 10/28 Verifying : python-linstor-1.23.0-1.xcpng8.3.noarch 11/28 Verifying : 1:java-11-openjdk-headless-11.0.23.0.9-2.el7_9.x86_64 12/28 Verifying : lksctp-tools-1.0.17-2.el7.x86_64 13/28 Verifying : http-nbd-transfer-1.6.0-1.xcpng8.3.x86_64 14/28 Verifying : linstor-client-1.23.0-1.xcpng8.3.noarch 15/28 Verifying : libxslt-1.1.28-5.el7.x86_64 16/28 Verifying : linstor-satellite-1.29.0-1.el7_9.noarch 17/28 Verifying : pcsc-lite-libs-1.8.8-8.el7.x86_64 18/28 Verifying : javapackages-tools-3.4.1-11.el7.noarch 19/28 Verifying : drbd-xen-9.28.0-1.el7.x86_64 20/28 Verifying : nbdkit-1.8.0-4.el7.x86_64 21/28 Verifying : drbd-9.28.0-1.el7.x86_64 22/28 Verifying : drbd-udev-9.28.0-1.el7.x86_64 23/28 Verifying : drbd-bash-completion-9.28.0-1.el7.x86_64 24/28 Verifying : linstor-controller-1.29.0-1.el7_9.noarch 25/28 Verifying : drbd-reactor-1.4.1-1.x86_64 26/28 Verifying : xcp-ng-linstor-1.2-2.xcpng8.3.noarch 27/28 Verifying : alsa-lib-1.1.4.1-2.el7.x86_64 28/28 Installed: xcp-ng-linstor.noarch 0:1.2-2.xcpng8.3 Dependency Installed: alsa-lib.x86_64 0:1.1.4.1-2.el7 copy-jdk-configs.noarch 0:3.3-10.el7_5 drbd.x86_64 0:9.28.0-1.el7 drbd-bash-completion.x86_64 0:9.28.0-1.el7 drbd-pacemaker.x86_64 0:9.28.0-1.el7 drbd-reactor.x86_64 0:1.4.1-1 drbd-udev.x86_64 0:9.28.0-1.el7 drbd-utils.x86_64 0:9.28.0-1.el7 drbd-xen.x86_64 0:9.28.0-1.el7 graphite2.x86_64 0:1.3.10-1.el7_3 harfbuzz.x86_64 0:1.3.2-1.el7 http-nbd-transfer.x86_64 0:1.6.0-1.xcpng8.3 java-11-openjdk-headless.x86_64 1:11.0.23.0.9-2.el7_9 javapackages-tools.noarch 0:3.4.1-11.el7 kmod-drbd.x86_64 0:9.2.11-1.1.xcpng8.3 libxslt.x86_64 0:1.1.28-5.el7 linstor-client.noarch 0:1.23.0-1.xcpng8.3 linstor-common.noarch 0:1.29.0-1.el7_9 linstor-controller.noarch 0:1.29.0-1.el7_9 linstor-satellite.noarch 0:1.29.0-1.el7_9 lksctp-tools.x86_64 0:1.0.17-2.el7 nbdkit.x86_64 0:1.8.0-4.el7 pcsc-lite-libs.x86_64 0:1.8.8-8.el7 python-javapackages.noarch 0:3.4.1-11.el7 python-linstor.noarch 0:1.23.0-1.xcpng8.3 python-lxml.x86_64 0:3.2.1-4.el7 tzdata-java.noarch 0:2024a-1.el7 Complete! [00:59 node-3 ~]# xe-toolstack-restart Executing xe-toolstack-restart done. [00:59 node-3 ~]# xe pool-join master-address=XXXXX master-username=XXXX master-password="YYYYY" Host agent will restart and attempt to join pool in 10.000 seconds... -
Backup fail whereas xostor cluster is "healthy"
Hello,
My post title may be missleading.
I built a pool with 3 nodes and 3 replicas.
- node1
- node2
- node3
When i remove node 3 ( linstor forget + xo forget node ).
The snapshot of disks are not possible anymore.
SR_BACKEND_FAILURE_1200(, local variable 'e' referenced before assignment, )
However, i have 2 healthy node on 3. it should work no ?
️ I don't need support, it's just an observation in our test pool, maybe a bug, i report Best regards