I have been investigating a network hardware offload issue with the Realtek RTL8125BG 2.5G on the same XCP-ng host computer. This separate issue was resolved by shutting off TX checksum offload on the physical interface.

It made me think that maybe the mouse problem is caused by something similar on the other NIC. The second NIC in the computer is Realtek R8111EPV (1G)(using Linux r8169 driver) which is the one I have been using when I noticed the mouse issue. So I ran a capture test on the XCP-ng host:

tcpdump -i eth1 -v -nn | grep incorrect

and found bad packet checksums on some of the received packets coming in to the Windows VM:

x.x.x.x.443 > [windows.vm.ip].49715: Flags [P.], cksum 0xc656 (incorrect -> 0xb854), seq 1:2433, ack 216, win 2052, length 2432
x.x.x.x.443 > [windows.vm.ip].49717: Flags [P.], cksum 0x2e6c (incorrect -> 0xcff2), seq 1:2921, ack 218, win 501, length 2920
x.x.x.x.443 > [windows.vm.ip].49720: Flags [.], cksum 0x1b11 (incorrect -> 0xb99c), seq 1:2921, ack 213, win 16384, length 2920
...

Running this on the XCP-ng host stopped the checksum errors:

ethtool -K eth1 gro off  # testing only, not persistent

Making this setting persistent:

xe pif-param-set uuid=my_eth1_PIF_UUID other-config:ethtool-gro="off"

After making this change I will monitor the VM to see if the mouse problem returns.

(In regards to uninstalling XenServer / Citrix Tools)

I tested uninstalling XenServer tools v9.4.0 on Windows and it seems to uninstall cleanly. (the xcp-ng docs mention there have been problems uninstalling everything cleanly in the past)

Also since XenServer tools v9.3.1 there is a new purge function:

uninstall.exe purge - (9.3.1 and later) Resets the VM to a clean state as it was before any part of the XenServer VM Tools were installed.

Full uninstall of all XenServer VM Tools components
https://docs.xenserver.com/en-us/citrix-hypervisor/vms/windows/vm-tools.html#full-uninstall-of-all-xenserver-vm-tools-components

What does the purge option remove?
https://docs.xenserver.com/en-us/citrix-hypervisor/vms/windows/vm-tools.html#what-does-the-purge-option-remove

@WayneSherman said in Mouse stops responding in XO console (XCP-ng 8.3, Win11 24H2):

It made me think that maybe the mouse problem is caused by something similar on the other NIC. The second NIC in the computer is Realtek R8111EPV (1G)(using Linux r8169 driver) which is the one I have been using when I noticed the mouse issue. So I ran a capture test ... and found bad packet checksums on some of the received packets coming in to the Windows VM

I cannot confirm that the network errors were related to the issue of the mouse freezing. After running all weekend (on eth1, with gro offloading set to on, and network packet errors present) I have not been able to replicate the mouse freezing.

@dinhngtu

Testing on a newly installed and updated Windows 7 Pro 24H2

Issue #1: attempting to install the certificate gives an error

1. open a command prompt as administrator
2. cd to the testsign directory
3. run this command:
   powershell -ExecutionPolicy Bypass .\install.ps1
4. After hitting enter for confirmation, I get these messages

Line 21 fails with Unauthorized Access Exception:

Import-Certificate -FilePath $PSScriptRoot\XCP-ng_Test_Signer.crt -CertStoreLocation Cert:\LocalMachine\TrustedPublisher

@WayneSherman said in Mouse stops responding in XO console (XCP-ng 8.3, Win11 24H2):

It made me think that maybe the mouse problem is caused by something similar on the other NIC. The second NIC in the computer is Realtek R8111EPV (1G)(using Linux r8169 driver) which is the one I have been using when I noticed the mouse issue. So I ran a capture test ... and found bad packet checksums on some of the received packets coming in to the Windows VM

I cannot confirm that the network errors were related to the issue of the mouse freezing. After running all weekend (on eth1, with gro offloading set to on, and network packet errors present) I have not been able to replicate the mouse freezing.

I have been investigating a network hardware offload issue with the Realtek RTL8125BG 2.5G on the same XCP-ng host computer. This separate issue was resolved by shutting off TX checksum offload on the physical interface.

It made me think that maybe the mouse problem is caused by something similar on the other NIC. The second NIC in the computer is Realtek R8111EPV (1G)(using Linux r8169 driver) which is the one I have been using when I noticed the mouse issue. So I ran a capture test on the XCP-ng host:

tcpdump -i eth1 -v -nn | grep incorrect

and found bad packet checksums on some of the received packets coming in to the Windows VM:

x.x.x.x.443 > [windows.vm.ip].49715: Flags [P.], cksum 0xc656 (incorrect -> 0xb854), seq 1:2433, ack 216, win 2052, length 2432
x.x.x.x.443 > [windows.vm.ip].49717: Flags [P.], cksum 0x2e6c (incorrect -> 0xcff2), seq 1:2921, ack 218, win 501, length 2920
x.x.x.x.443 > [windows.vm.ip].49720: Flags [.], cksum 0x1b11 (incorrect -> 0xb99c), seq 1:2921, ack 213, win 16384, length 2920
...

Running this on the XCP-ng host stopped the checksum errors:

ethtool -K eth1 gro off  # testing only, not persistent

Making this setting persistent:

xe pif-param-set uuid=my_eth1_PIF_UUID other-config:ethtool-gro="off"

After making this change I will monitor the VM to see if the mouse problem returns.

@Andrew said in Mouse stops responding in XO console (XCP-ng 8.3, Win11 24H2):

I don't see this as an XO problem as I use the same XO for other hosts/guests that don't have a problem.

I agree, I don't think it is an XO problem. When the mouse stopped working I rebooted XO but the mouse was still broke. I restarted the toolstack on XCP-ng and it was still broke. I think XO is sending mouse events but the Windows VM has stopped receiving or responding to them.

(also, the problem doesn't happen on the Debian 12 VM installed on the same host)

@WayneSherman said in Mouse stops responding in XO console (XCP-ng 8.3, Win11 24H2):

In the XO console, the guests mouse stops responding after some random amount of time (e.g. 5 min or 20 min).

The mouse freezing was becoming more frequent and happening sooner after a VM reboot (sometimes seconds after I start using the web UI console, or within a minute or two).

So I shutdown and rebooted the XCP-ng host computer and the mouse freezing does not appear to happen quickly now (hasn't happened yet). I hypothesize that the longer the host computer is running the faster and more frequent this problem happens.

@dinhngtu said in Issues with XCP-ng Windows PV Tools 9.0.9000.0 Testsign Pre-release:

With 9.0.9000.0, you could sometimes run into this issue if you reboot too early after installing. This will also be fixed in the next release.

I just did two more tests and the boot-up failure happened both times. On the last test I waited some time (like 30 to 60 sec) after the driver installation before rebooting (using Windows restart from the start menu). Even though I waited before rebooting I still encountered the boot-up problem:

Windows is spinning on this screen and doesn't finish booting:

@dinhngtu said:

you could use the following command to install the certificate manually:

certutil -addstore -f TrustedPublisher XCP-ng_Test_Signer.crt

Yes that command works without any errors

@dinhngtu

Issue #2
I cannot replicate. Yesterday after installing the drivers and rebooting, Windows never finished booting up. It just gave me the spinning icon.. Today after installing the PV drivers it is booting up correctly.

@dinhngtu

Testing on a newly installed and updated Windows 7 Pro 24H2

Issue #1: attempting to install the certificate gives an error

1. open a command prompt as administrator
2. cd to the testsign directory
3. run this command:
   powershell -ExecutionPolicy Bypass .\install.ps1
4. After hitting enter for confirmation, I get these messages

Line 21 fails with Unauthorized Access Exception:

Import-Certificate -FilePath $PSScriptRoot\XCP-ng_Test_Signer.crt -CertStoreLocation Cert:\LocalMachine\TrustedPublisher

In XO in the advanced settings for a Windows 11 VM, "Viridian" is enabled. If I disable it, it does correctly disable that platform option on XCP-ng host (which I confirmed with the command: xe vm-param-get uuid=xxx param-name=platform). But when booting up or shutting down the VM, the Viridian setting reverts back to Enabled.

Is this a bug, or is there another way to make the change permanent?

Testing:

• XCP-ng v8.3 (fresh install from downloaded ISO and fully updated)
• Xen Orchestra (installed and built from git (2024-Nov-18 9939cee)
• Guest VM - Windows 11 Pro 24H2 (fresh install from downloaded ISO and fully updated)
  (tested without guest tools and with XenTools 9.40 with the same behavior)

In the XO console, the guests mouse stops responding after some random amount of time (e.g. 5 min or 20 min). When this happens, the keyboard still works in the XO console and the VM is still responsive. The mouse still works when remote controlling the VM via remote support client (and I can see it move in the console when controlling it this way). Rebooting the XO server does not fix it, Restarting the XCP-ng toolstack does not fix it. Rebooting Windows does fix it until it happens again at some random interval.