Might a different solution be to use a USB network bridge instead of direct attached USB? Something like this https://www.seh-technology.com/products/usb-deviceserver/utnserver-pro.html (There are different options available)... We use my-utn-50a with hardware USB keys and it has shown to be very reliable over the years.

AFAIK, you have to do manual passthrough for USB so if you want to attach ethernet dongle to a VM, you plug it in the host and turn off the VM, passthrough the USB to the VM, start the VM, never do snapshot of this VM as it will fail.