XCP-ng 8.3 updates announcements and testing
-
Hello,
I deleted the entire XOA and downloaded fresh image, recreated configuration and blank SR.
The issue is gone. I will test some VDI migrations this week.There are is some performance impact and timeouts when creating 100-200 VMs on single 10 TB SR. I use xo-cli vm.create script. See you later
-
-
Congrats to everyone on those, it was a huge amount of work, test, and as usual, great community feedback!
-
Hello. just wanted to let you know that I got this error message on one of my servers (the others worked fine), when installing the last 25 updates manually via "yum update"
[10:39 xen1 ~]# yum update Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile Excluding mirror: updates.xcp-ng.org * xcp-ng-base: mirrors.xcp-ng.org Excluding mirror: updates.xcp-ng.org * xcp-ng-updates: mirrors.xcp-ng.org Resolving Dependencies --> Running transaction check ---> Package blktap.x86_64 0:3.55.5-6.3.xcpng8.3 will be updated ---> Package blktap.x86_64 0:3.55.5-6.7.xcpng8.3 will be an update ---> Package forkexecd.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package forkexecd.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package kernel.x86_64 0:4.19.19-8.0.46.1.xcpng8.3 will be updated ---> Package kernel.x86_64 0:4.19.19-8.0.46.2.xcpng8.3 will be an update ---> Package message-switch.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package message-switch.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package qcow-stream-tool.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package qcow-stream-tool.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package rrdd-plugins.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package rrdd-plugins.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package sm.x86_64 0:3.2.12-17.1.xcpng8.3 will be updated ---> Package sm.x86_64 0:3.2.12-17.7.xcpng8.3 will be an update ---> Package sm-cli.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package sm-cli.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package sm-fairlock.x86_64 0:3.2.12-17.1.xcpng8.3 will be updated ---> Package sm-fairlock.x86_64 0:3.2.12-17.7.xcpng8.3 will be an update ---> Package squeezed.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package squeezed.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package varstored-guard.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package varstored-guard.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package vhd-tool.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package vhd-tool.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package wsproxy.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package wsproxy.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xapi-core.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xapi-core.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xapi-nbd.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xapi-nbd.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xapi-rrd2csv.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xapi-rrd2csv.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xapi-storage-script.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xapi-storage-script.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xapi-tests.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xapi-tests.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xapi-xe.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xapi-xe.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xcp-networkd.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xcp-networkd.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xcp-ng-pv-tools.noarch 0:8.3-16.xcpng8.3 will be updated ---> Package xcp-ng-pv-tools.noarch 0:8.3-17.xcpng8.3 will be an update ---> Package xcp-rrdd.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xcp-rrdd.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xenopsd.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xenopsd.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xenopsd-cli.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xenopsd-cli.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package xenopsd-xc.x86_64 0:26.1.3-1.9.xcpng8.3 will be updated ---> Package xenopsd-xc.x86_64 0:26.1.3-1.10.xcpng8.3 will be an update ---> Package zabbix-agent2.x86_64 0:7.0.25-release1.el7 will be updated ---> Package zabbix-agent2.x86_64 0:7.0.26-release1.el7 will be an update ---> Package zabbix-agent2-plugin-ember-plus.x86_64 0:7.0.25-release1.el7 will be updated ---> Package zabbix-agent2-plugin-ember-plus.x86_64 0:7.0.26-release1.el7 will be an update ---> Package zabbix-agent2-plugin-mongodb.x86_64 0:7.0.25-release1.el7 will be updated ---> Package zabbix-agent2-plugin-mongodb.x86_64 0:7.0.26-release1.el7 will be an update ---> Package zabbix-agent2-plugin-mssql.x86_64 0:7.0.25-release1.el7 will be updated ---> Package zabbix-agent2-plugin-mssql.x86_64 0:7.0.26-release1.el7 will be an update ---> Package zabbix-agent2-plugin-postgresql.x86_64 0:7.0.25-release1.el7 will be updated ---> Package zabbix-agent2-plugin-postgresql.x86_64 0:7.0.26-release1.el7 will be an update --> Finished Dependency Resolution Dependencies Resolved ==================================================================================================================================================================================================================== Package Arch Version Repository Size ==================================================================================================================================================================================================================== Updating: blktap x86_64 3.55.5-6.7.xcpng8.3 xcp-ng-updates 772 k forkexecd x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 2.5 M kernel x86_64 4.19.19-8.0.46.2.xcpng8.3 xcp-ng-updates 29 M message-switch x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 4.6 M qcow-stream-tool x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 1.4 M rrdd-plugins x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 9.8 M sm x86_64 3.2.12-17.7.xcpng8.3 xcp-ng-updates 391 k sm-cli x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 4.8 M sm-fairlock x86_64 3.2.12-17.7.xcpng8.3 xcp-ng-updates 52 k squeezed x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 1.9 M varstored-guard x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 4.9 M vhd-tool x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 4.9 M wsproxy x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 1.1 M xapi-core x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 30 M xapi-nbd x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 3.1 M xapi-rrd2csv x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 3.0 M xapi-storage-script x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 2.5 M xapi-tests x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 5.0 M xapi-xe x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 1.5 M xcp-networkd x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 4.8 M xcp-ng-pv-tools noarch 8.3-17.xcpng8.3 xcp-ng-updates 16 M xcp-rrdd x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 3.6 M xenopsd x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 1.4 M xenopsd-cli x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 2.0 M xenopsd-xc x86_64 26.1.3-1.10.xcpng8.3 xcp-ng-updates 5.3 M zabbix-agent2 x86_64 7.0.26-release1.el7 zabbix 6.6 M zabbix-agent2-plugin-ember-plus x86_64 7.0.26-release1.el7 zabbix 1.6 M zabbix-agent2-plugin-mongodb x86_64 7.0.26-release1.el7 zabbix 3.8 M zabbix-agent2-plugin-mssql x86_64 7.0.26-release1.el7 zabbix 2.7 M zabbix-agent2-plugin-postgresql x86_64 7.0.26-release1.el7 zabbix 3.1 M Transaction Summary ==================================================================================================================================================================================================================== Upgrade 30 Packages Total size: 162 M Is this ok [y/d/N]: y Downloading packages: Running transaction check Running transaction test Transaction check error: file /usr/bin/qemu-img from install of blktap-3.55.5-6.7.xcpng8.3.x86_64 conflicts with file from package qemu-img-10:1.5.3-175.el7_9.6.x86_64 Error Summary ------------- [10:39 xen1 ~]#So in consequence the update seems to simply fail. I am aware that the manually installed zabbix packages are not "official" packages. However, from my current understanding the error looks more like a non-zabbix related issue. I am wondering, since I have another exactly identical server, which updated fine. If you need some informations, just let me know.
-
Sorry, about this. I think when migrating PV VMs from a previous system I had issues and "needed" to install "qemu-img" and I did not remember. So removing it, solved the problem. So all my fault. Sorry about this message!!!
-
Ran the 2 updates released today and...
Back to only showing one VM inBackup/Restoreas it did a month or 2 ago.
Ran the replication job and all VMs showed up inBackup/Restoreagain. (XO5) -
anyone know if applying these two patches requires rebooting?
-
@marcoi I've read in the blog post of Vates that no.
Did it anyway. -
Indeed, no reboot required if those are the only patches that you are applying, as indicated in the blog post.
-
@stormi I getting a response to reboot after applying to master.
-
XO isn't subtle and will ask for reboot regardless of the type of update (because for now there's no way to know if reboot is required)
-
@olivierlambert lol okay got it
-
I got the results. You can access our lab, feel free, nothing is productive. Support ID: 40797
High count of VDI / VMs on single SR make entire Storage slow.
Migration fails because of:
[21:52 xcp-ng-1 ~]# xe sr-scan uuid=43dbbe8e-039a-e66c-f6cb-d88de4f4d962
Error code: SR_BACKEND_FAILURE_1200
Error parameters: , list index out of range,59 Lines of log:
May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] ['/sbin/lvchange', '-an', '/dev/VG_XenStorage-43dbbe8e-039a-e66c-f6cb-d88de4f4d962/QCOW2-ffcf4c40-0d81-4da7-9908-478629c88358'] May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] pread SUCCESS May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper released May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper acquired May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper sent '56171 - 832.417448266' May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] ['/sbin/dmsetup', 'status', 'VG_XenStorage--43dbbe8e--039a--e66c--f6cb--d88de4f4d962-QCOW2--ffcf4c40--0d81--4da7--9908--478629c88358'] May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] pread SUCCESS May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper released May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] lock: released /var/lock/sm/lvm-43dbbe8e-039a-e66c-f6cb-d88de4f4d962/ffcf4c40-0d81-4da7-9908-478629c88358 May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper acquired May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper sent '56171 - 832.433077125' May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] ['/sbin/vgs', '--noheadings', '--nosuffix', '--units', 'b', 'VG_XenStorage-43dbbe8e-039a-e66c-f6cb-d88de4f4d962'] May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] pread SUCCESS May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper released May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper acquired May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper sent '56171 - 832.64258113' May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] ['/sbin/vgs', '--readonly', 'VG_XenStorage-43dbbe8e-039a-e66c-f6cb-d88de4f4d962'] May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] pread SUCCESS May 8 21:40:34 xcp-ng-2 fairlock[8031]: /run/fairlock/devicemapper released May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] lock: released /var/lock/sm/43dbbe8e-039a-e66c-f6cb-d88de4f4d962/sr May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] ***** generic exception: sr_scan: EXCEPTION <class 'IndexError'>, list index out of range May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/SRCommand.py", line 113, in run May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] return self._run_locked(sr) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/SRCommand.py", line 163, in _run_locked May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] rv = self._run(sr, target) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/SRCommand.py", line 377, in _run May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] return sr.scan(self.params['sr_uuid']) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/LVMoHBASR", line 163, in scan May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] LVMSR.LVMSR.scan(self, sr_uuid) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/LVMSR.py", line 822, in scan May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] new_vdi = self.vdi(cbt_uuid) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/LVMoHBASR", line 230, in vdi May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] return LVMoHBAVDI(self, uuid) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/VDI.py", line 100, in __init__ May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] self.load(uuid) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/LVMSR.py", line 1375, in load May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] size = int(self.sr.srcmd.params['args'][0]) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] ***** LVM over FC: EXCEPTION <class 'IndexError'>, list index out of range May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/SRCommand.py", line 392, in run May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] ret = cmd.run(sr) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/SRCommand.py", line 113, in run May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] return self._run_locked(sr) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/SRCommand.py", line 163, in _run_locked May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] rv = self._run(sr, target) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/SRCommand.py", line 377, in _run May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] return sr.scan(self.params['sr_uuid']) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/LVMoHBASR", line 163, in scan May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] LVMSR.LVMSR.scan(self, sr_uuid) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/LVMSR.py", line 822, in scan May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] new_vdi = self.vdi(cbt_uuid) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/LVMoHBASR", line 230, in vdi May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] return LVMoHBAVDI(self, uuid) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/VDI.py", line 100, in __init__ May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] self.load(uuid) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] File "/opt/xensource/sm/LVMSR.py", line 1375, in load May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] size = int(self.sr.srcmd.params['args'][0]) May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] May 8 21:40:34 xcp-ng-2 SM: [56171][MainThread] lock: closed /var/lock/sm/lvm-43dbbe8e-039a-e66c-f6cb-d88de4f4d962/030878c2-49fd-4b0f-a971-9dfa121249a3Best regards
Igor
-
@olivierlambert Seeing some failure/errors on CR jobs. It leaves VDIs attached to Control Domain... Next run it normally works. I have not seen this error until after the current sm update. Running XO (commit 7e144).
"message": "INTERNAL_ERROR(Storage_error ([S(Illegal_transition); [[S(Activated);S(RO)];[S(Activated);S(RW)]]]))", "name": "XapiError", "stack": "XapiError: INTERNAL_ERROR(Storage_error ([S(Illegal_transition);[[S(Activated);S(RO)];[S(Activated);S(RW)]]]))\n at XapiError.wrap (file:///opt/xo/xo-builds/xen-orchestra-202605050700/packages/xen-api/_XapiError.mjs:16:12)\n at default (file:///opt/xo/xo-builds/xen-orchestra-202605050700/packages/xen-api/_getTaskResult.mjs:13:29)\n at Xapi._addRecordToCache (file:///opt/xo/xo-builds/xen-orchestra-202605050700/packages/xen-api/index.mjs:1078:24)\n at file:///opt/xo/xo-builds/xen-orchestra-202605050700/packages/xen-api/index.mjs:1112:14\n at Array.forEach (<anonymous>)\n at Xapi._processEvents (file:///opt/xo/xo-builds/xen-orchestra-202605050700/packages/xen-api/index.mjs:1102:12)\n at Xapi._watchEvents (file:///opt/xo/xo-builds/xen-orchestra-202605050700/packages/xen-api/index.mjs:1275:14)\n at process.processTicksAndRejections (node:internal/process/task_queues:104:5)" -
Ping @Team-Storage
-
Hello @igorglock, Damien is in holiday this week but he identified the issue and a patch should be tested for the next release of SM.
-
I get this in dmesg after the latest updates :
[ 54.673443] python3[3691]: segfault at 200000 ip 00007f16eb8eca9f sp 00007ffd b84e9ff0 error 4 in libpython3.6m.so.1.0[7f16eb804000+28d000] [ 54.673450] Code: 01 00 00 8d 5f ff 48 8d 2d de 3a 3c 00 c1 eb 03 44 8d 24 1b 4e 8b 44 e5 00 49 8b 70 10 49 39 f0 74 5f 49 8b 40 08 41 83 00 01 <48> 8b 38 48 85 ff 49 89 78 08 74 0d 48 83 c4 10 5b 5d 41 5c c3 0f [ 84.587661] xapi[3697]: segfault at 7f28cacaea40 ip 00007f28c6df0ec2 sp 00007 f289a5b8af0 error 6 in libjemalloc.so.2[7f28c6d85000+85000] [ 84.587669] Code: 48 2b 73 08 44 8b 4d 84 ba 01 00 00 00 49 83 c2 01 49 0f af f1 4c 8d 0d ac 72 42 00 48 89 f1 48 c1 ee 26 48 c1 e9 20 48 d3 e2 <48> 31 54 f3 40 48 8b 8d 58 ff ff ff 48 8b 33 48 8d be 00 00 00 10 -
@Andrew I have experienced this twice so far as well.
-
@stormi I'm also getting error on some VMs while trying to export a disk and also trying to even start some VMs from NFS (that were fine before).
xo-server[565]: 2026-05-13T02:53:15.746Z xo:api WARN admin | vm.start(...) [2s] =!> XapiError: INTERNAL_ERROR(xenopsd internal error: Storage_error ([S(Illegal_transition);[[S(Activated);S(RO)];[S(Activated);S(RW)]]])) xo-server[565]: 2026-05-13T02:53:40.652Z xo:api WARN admin | vm.start(...) [3s] =!> XapiError: SR_BACKEND_FAILURE_46(, The VDI is not available [opterr=VDI 399734eb-5965-4799-ac36-f6dd774db867 not detached cleanly], ) -
Ping @Team-Storage again, about the last comment.
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login