The Xen / XCP-ng equivalent for VMXNET are Xen PV device drivers which either come from Citrix Xenserver, XCP-ng or the Xenserver project itself. Check the Wiki. It describes the installation pretty well:
https://xcp-ng.org/docs/guests.html#windows

As for 3D acceleration within the VM there is hardly a comparable option unless you do some PCIe pass through of a GPU.