XSA-468: multiple Windows PV driver vulnerabilities - update now!

Forza

Hi,

It is not clear to me if the old XCP-ng PV drivers (8.2.2.200-RC1) are affected or not. How should we proceed if they are? AFAIK it is no easy task to migrate to WindowsUpdate drivers and it usually ends up with some issue.

dinhngtu

@Forza Yes, all XCP-ng drivers before 9.0.9065 are affected, including 8.2.x.

There's no need to enable the "Manage Citrix PV drivers via Windows Update" option. You should remove the existing drivers with XenClean and install XenServer drivers 9.4.1 or later. (See the "How to patch my VMs?" section for detailed instructions)

If updating is absolutely not an option, you can try the mitigation script, but be aware that it's only a bandaid and not meant to be a long-term or complete fix.

flakpyro

Until the XO update is released by Vates you can filter your PV Driver version using the following:

pvDriversVersion:!9.4.1

This will show you all VMs not running that version. If you have Linux VMs they will be shown in this filter. If you have tagged your windows VMs you can combine this with a tag for example:

power_state:running pvDriversVersion:!9.4.1 tags:/^Windows$/

Remove the ! to filter for VMs that are running the latest 9.4.1 version.

dinhngtu

Hi all,

Xen Orchestra 5.107.0 (latest channel) and 5.106.4 (stable channel) have been released. They contain the automatic alert and vulnerable? filter for Windows VMs affected by XSA-468.

conitrade-as

After upgrading the VM tools the mitigation script still shows vulnerable devices on some hosts. After running the script to apply the mitigations, nothing is reported as being vulnerable anymore.

Thus the question: Is applying the mitigations a necessary action as well? Or does installing the v9.4.1 tools fix the vulnerability?

dinhngtu

@conitrade-as Installing the XenServer 9.4.1 tools and rebooting (a couple times) will be enough. Do your VMs show as vulnerable (red) or unable to detect (orange)? There are some new host XAPI updates for correcting driver version reporting that are needed for the XO feature to work.

conitrade-as

@dinhngtu Ok, I will keep that in mind as I go through all the VMs. As I currently cannot update XCP-ng on all hosts (8.2.1 LTS), the VMs where the new tools were installed and mitigations applied show up as "orange".

On a XCP-ng 8.3 test hosts with all updates applied the detection works as advertised.

DustinB

@dinhngtu said in XSA-468: multiple Windows PV driver vulnerabilities - update now!:

@conitrade-as Installing the XenServer 9.4.1 tools and rebooting (a couple times) will be enough.

The multiple reboots is the annoying part, ha... working on my systems now. I've always used https://www.xenserver.com/downloads for this.

On numerous VM's I've been installing over top of the existing drivers and within System Manager I still see

.

Should I just uninstall the old stuff first, then install the updated to address?

dinhngtu

@conitrade-as The XAPI update was backported to 8.2.1 LTS so you could update your hosts for the new display.

@DustinB If you're coming from old XenServer tools to new, no need to uninstall.

DustinB

@dinhngtu said in XSA-468: multiple Windows PV driver vulnerabilities - update now!:

@DustinB If you're coming from old XenServer tools to new, no need to uninstall.

Interesting, the alert is still within XOA

...

dinhngtu

@DustinB Do you have the newest pool updates? The warning depends on that.

DustinB

@dinhngtu said in XSA-468: multiple Windows PV driver vulnerabilities - update now!:

@DustinB Do you have the newest pool updates? The warning depends on that.

I have an open ticket for that.....

conitrade-as

@dinhngtu On a Windows 10 VM rebooting alone did not do the trick. After 5 reboots the script still reports vulnerable devices:

.\Install-XSA468Workaround.ps1 -Scan

Looking for vulnerable XenIface objects
Found vulnerable object XENBUS\VEN_XSC000&DEV_IFACE\_
Found vulnerable object XENBUS\VEN_XSC000&DEV_IFACE\_

Looking for vulnerable XenIface WMI GUIDs
Found vulnerable WMI GUID 1D80EB99-A1D6-4492-B62F-8B4549FF0B5E
Found vulnerable WMI GUID 12138A69-97B2-49DD-B9DE-54749AABC789
Found vulnerable WMI GUID AB8136BF-8EA7-420D-ADAD-89C83E587925

Found XenIface vulnerability, it's recommended to run the script
True

Running .\Install-XSA468Workaround.ps1 works as expected. After another reboot nothing is reported as being vulnerable anymore.

On a Windows 2019 Server I saw the behaviour you described: Installing the tools and a reboot was enough.

conitrade-as

On another Windows 10 host it worked. What was different: I saw the message box "Tools have been installed successfully". May be that makes a difference?

dinhngtu

@conitrade-as I saw that on your Windows 10 VM, the "Manage Citrix PV drivers via Windows Update" option is enabled. That one might have needed a Windows Update to install the fixed drivers. Do you have that option enabled in other VMs?

conitrade-as

@dinhngtu On the machine where it worked, the option "Manage Citrix PV drivers via Windows Update" was not enabled. Seems that my older BIOS Windows 10 VMs have that option enabled. On all UEFI VMs the options is disabled.

As I wanted to go and check that is present in the templates, I realized that the Windows Templates are gone from Xen Orchestra v5.106.4???

DustinB

@conitrade-as said in XSA-468: multiple Windows PV driver vulnerabilities - update now!:

As I wanted to go and check that is present in the templates, I realized that the Windows Templates are gone from Xen Orchestra v5.106.4??

Can confirm all templates for Windows are missing on 5.107.0 as well.

dinhngtu

@conitrade-as @DustinB Thanks, reported the template issue to XO team.

conitrade-as

Here is another interesting fact: After installing the new tools (v.9.4.1) my static routes in Windows were all gone. Definitively a good way to loose connectivity to your domain controller. And that's why you have good monitoring and store things in Ansible et al. ...

DustinB

@conitrade-as said in XSA-468: multiple Windows PV driver vulnerabilities - update now!:

Here is another interesting fact: After installing the new tools (v.9.4.1) my static routes in Windows were all gone. Definitively a good way to loose connectivity to your domain controller. And that's why you have good monitoring and store things in Ansible et al. ...

Statically assign, but keep your DHCP server with reservations to address these types of issues