XCP-ng 8.3 updates announcements and testing

semarie

"xenopsd internal error: Hotplug.Hotplug_error(\"Failed to read /xapi/450015b0-e878-5067-e8b6-000000000001/private/vif/1/trunks\")"

This is related to new code in the upgrade. the trunks attribute is used for VLAN filtering (new feature).

it seems that the running code is the new one (it is knowning about trunks) but the VM doesn't have the trunks attribute.

I would be interested to have a clean picture of the situation:

on which host this VM is resident on
version of the master
version of the host where the VM is resident

flakpyro

@semarie In my case the pool i tested these updates on was a 2 host pool, i patched each host using yum and then kicked off a rolling pool reboot. So the VMs would have been resident on the pool master since it is the first rebooted when running a rolling pool reboot. I was unable to manually migrate the VMs resident on this host as well, attempting to do so would show the same error.

In my case though restarting the tool stack on both hosts allowed migrations to complete and a rolling pool reboot to run successfully.

gduperrey

@acebmxer This error does not appear to be linked to the update, but rather to a repo that has not yet had time to synchronize or is in the process of doing so at the time of your update.

semarie

@flakpyro it seems to me the proper upgrade path is:

put the master in maintenance mode (xe host-disable host=$MASTER)
evacuate the master (xe host-evacuate host=$MASTER)
yum update the master
reboot the master (xe host-reboot host=$MASTER)
once done, do the same of the others hosts

the VM would have been updated with the new trunks attribute when migrating to some updated host (in your case, when migrating to the master).

flakpyro

@semarie If that's the case then the update will likely go smoothly once its released to the stable repos and users use Xen Orchestra to run a rolling pool update which follows that workflow. Since this was a test pool and using the test repos i ran the update via yum without evacuating the host first.

semarie

@flakpyro yes. thanks for your test and to reporting the problem anyway. it is helping us to see what kind of problems users could have.

gduperrey

Thank you everyone for your tests and your feedback!

The updates are live now: https://xcp-ng.org/blog/2026/08/18/august-2026-updates-1-for-xcp-ng-8-3-lts/

Andrew

@gduperrey Rolling pool update worked with released production patches.

MajorP93

Hello, so I got access to the test environment back and was able to install this set of patches.

I actually installed them on the XCP-ng host before you guys released them to the stable repository.

I ran:

yum clean metadata --enablerepo=xcp-ng-testing,xcp-ng-candidates
yum update --enablerepo=xcp-ng-testing,xcp-ng-candidates

and rebooted the system.

Unfortunately I have to say that this is the first time that patches broke my system.

While the XCP-ng host is still able to boot, it is not longer able to mount my SRs.

I have 2 SR in this test environment:

1x Linstor vSAN iSCSI configured as QCOW2
1x TrueNAS Core NFSv3 configured as VHD

I spent some hours troubleshooting this and appearently it is caused by jumbo frames no longer working after applying these XCP-ng patches.

The hypervisor / storage network in this testing environment is using jumbo frames everywhere (all switches involved, all storage systems).
Prior to installing the updates everything was working fine.

Now I can not ping the storage systems anymore using jumbo frames.
(ping -M do -s 8972 ...)

Hence the tasks that are meant to mount the SRs are stuck forever:

[14:38 xcpng-test01 ~]# xe task-list
uuid ( RO)                : a6ba1324-bfe1-8aca-c5d1-d7cc57d37cca
          name-label ( RO): PBD.plug
    name-description ( RO): 
              status ( RO): pending
            progress ( RO): 0.000


uuid ( RO)                : f9c76606-e516-553f-706c-ff52bc303e2d
          name-label ( RO): PBD.plug
    name-description ( RO): 
              status ( RO): pending
            progress ( RO): 0.000

The interesting thing is that "ip a" is still showing MTU 9000:

[14:44 xcpng-test01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
    link/ether ac:1f:6b:ad:2c:b2 brd ff:ff:ff:ff:ff:ff
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
    link/ether ac:1f:6b:ad:2c:b3 brd ff:ff:ff:ff:ff:ff
4: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 80:61:5f:10:85:87 brd ff:ff:ff:ff:ff:ff
5: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
    link/ether ec:0d:9a:8c:00:fc brd ff:ff:ff:ff:ff:ff
6: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master ovs-system state DOWN group default qlen 1000
    link/ether ec:0d:9a:8c:00:fd brd ff:ff:ff:ff:ff:ff
7: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 4a:1c:22:40:4a:d2 brd ff:ff:ff:ff:ff:ff
8: xenbr2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether ec:0d:9a:8c:00:fc brd ff:ff:ff:ff:ff:ff
9: xenbr3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether ec:0d:9a:8c:00:fd brd ff:ff:ff:ff:ff:ff
10: xenbr4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 80:61:5f:10:85:87 brd ff:ff:ff:ff:ff:ff
11: xenbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether ac:1f:6b:ad:2c:b3 brd ff:ff:ff:ff:ff:ff
12: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether ac:1f:6b:ad:2c:b2 brd ff:ff:ff:ff:ff:ff
13: xapi1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 80:61:5f:10:85:87 brd ff:ff:ff:ff:ff:ff
    inet 10.10.160.24/24 brd 10.10.160.255 scope global xapi1
       valid_lft forever preferred_lft forever

Can't start XO VM right now as it lives on one of the SR but XO Lite is also still showing jumbo frames being enabled:

Was something changed in this set of patches that could cause this issue?

Thanks and best regards

bufanda

@gduperrey said:

Thank you everyone for your tests and your feedback!

The updates are live now: https://xcp-ng.org/blog/2026/08/18/august-2026-updates-1-for-xcp-ng-8-3-lts/

Installed on my prod pool and instantly got an SMART error. What a timing But other than that updates working withut issues.

anthoineb

@MajorP93 Try first to fix the ping issue with jumboframe before anything. If the issue persists then:
what are the types of your SRs unable to be mounted? Is there any error in SMlog or xensouce.log?

gduperrey

Hello @MajorP93,

Several things in your message raise questions for me.

First, this SR:

1x Linstor vSAN iSCSI configured as QCOW2

What is this storage and how is it configured?
Currently, we do not support QCOW2 on Linstor/Xostor. So, is it simply an iSCSI device to which you have applied the QCOW2 format, or is it a Xostor on the XCP-ng pool?

Regarding jumbo frames, we do not use them internally and are not currently testing them. It is often recommended not to use them for the management interface, as indicated in the XCP-ng documentation: https://docs.xcp-ng.org/networking/

Non-standard MTUs (such as jumbo frames) are not supported on management interfaces. Using them can lead to serious issues, including failed pool member joins or unexpected network outages.

Is that the case here?

bleader

@majorp93 no, these updates should not change anything on the MTU behavior. From your screenshots and ip a output, I assume eth4 is the only link used, than your vlan1060 network is indeed your management network. I guess with some confidence that:

eth4 is your actual NIC
xenbr4 is the pool-wide network without vlan using eth4
xapi1 is the bridge for vlan1060
If that's right indeed on your host side, the MTUs are properly set.

As Gaël said, jumbo on management network is not officially supported, because we always end up in situation similar to yours, where something stops working for "some" reason

But as you said, when everything is setup properly, it does work.

I have test hosts at home up to date but no similar setup to yours for SR. I tried with a pool-wide and a pool-wide + vlan as management with 9000 and the ping -M do -s 8972 does work fine in both cases, so nothing I can see here.

When it fails, what do you see? Message too long? mtu=1500 ? something else?

Do you have multiple hosts in that pool? Can you try that between the hosts as well and not toward the storage systems? I would also check on ovs with ovs-vsctl list interface that each of eth4, xennbr4 and xapi1 report 9000 mtu.

I know you said it was properly setup everywhere, but I would still be tempted to think there is a setup issue somewhere.