9.2.385 snapshot/backup corruption fix: which versions were affected, and does it apply to XenServer VM Tools?

trelane

The 9.2.385 release notes list "Fix data corruption issue during in-memory snapshots and backups," with no detail on scope. Two questions before I decide what to do with my Windows VMs:

Which XCP-ng tools versions had the bug? Was it only 9.2.350, or does 9.1.x have it too? I just upgraded a VM from 9.1.200 and want to know whether I should re-check older backups of it.
Does it affect XenServer VM Tools? I have a Server 2025 VM on XenServer VM Tools 9.6.0 and would rather not swap tool families on it unless the bug applies.

Thanks.

poddingue

I went looking and couldn't find the answer either.

The 9.2.385 changelog puts that fix under "Changes since 9.2.350", so the fix isn't in 9.2.350, but as far as I can see that says nothing about 9.1.x either way.

The driver changes between the two tags are submodule bumps, and I can't read those well enough to tell where the bug came from.

XenServer VM Tools are a separate build from Citrix as far as I know, so I wouldn't assume anything for your 9.6.0 VM.

This needs someone who worked on the fix, so maybe @Team-Hypervisor-Kernel can say which versions had the bug and whether older backups are worth re-checking.

dinhngtu

@trelane Hello, the bug affects Xen drivers of any 9.0 series (considering the upstream code, it goes back to circa September 2019). 9.2.385 contains the fix, which was merged upstream just this month. The bug also impacts network connections, however to a lesser extent due to its nature.

I believe that the bug is also present in XenServer VM Tools 9.6.0 and by extension, older versions of it.

We recommend that you switch to the XCP-ng tools whenever possible, also due to its other benefits (fully open source, validated for XCP-ng and optimized for Xen Orchestra).

cg

@trelane Seems to be a pretty annoying task. It says Upgrade from 9.2.350 is blocked, Need to apply "9.2.351 compatibility update" first.

I just read the release notes which are buggy:
Important note for users upgrading from 9.2.350: You must first install the included XenTools-fix-9.2.351.msp hotfix to upgrade to 9.2.351 before installing 9.2.385.

The zip-file does not containe that msp!
...it's a separate file to download.

trelane

@cg Agreed. The next person can grab the .msp here:
https://github.com/xcp-ng/win-pv-drivers/releases/download/v9.2.385/XenTools-fix-9.2.351.msp

@dinhngtu Could the release note say "attached to this release" instead of "included"? It reads as if the .msp ships inside the zip. And thanks, your answer settled it. Both of my Windows VMs are on 9.2.385 now, and I took fresh full backups after the switch.

When I turned off "Manage Citrix PV drivers via Windows Update" (has-vendor-device=false) to switch from XenServer VM Tools, Windows treated the NICs as new adapters. The static IPs were gone and one NIC came back on the Public firewall profile. Have console access ready, or a startup script that puts the IPs back by MAC address.

cg

@trelane said:
When I turned off "Manage Citrix PV drivers via Windows Update" (has-vendor-device=false) to switch from XenServer VM Tools, Windows treated the NICs as new adapters.

That bug had already cost me so much time. It has a looong history and seems to be "nailed in" when the VM had old tools installed at some point.
I didn't have that problem with newer VMs that never had old tools tho.

dinhngtu

@cg @trelane I've updated the release notes and added a link to the 9.2.351 patch.

For the IP loss bug when toggling the "Manage Citrix PV..." setting, this bug should not happen on XCP-ng tools. In theory, it should not happen on the XenServer 9.6.0 drivers either, but I haven't tested.

If you need to reconfigure your VM's IP, check out the commands xe vif-configure-ipv4/ipv6. Currently it supports configuring IP/prefix/gateway; DNS configuration is coming soon.

trelane

@dinhngtu Thanks for fixing the note.

The IP loss did happen on 9.6.0. On the first boot after I set has-vendor-device=false, the guest still reported tools 9.6.0 build 19 (xenbus 9.1.13.23, xenvif 9.1.15.8, xennet 9.1.8.6), and both static IPs were already gone: the main NIC was on DHCP and the second NIC had a 169.254 address. XenClean and the 9.2.385 install came after that.