@timemaster5 Ah very cool, thank you for looking into this and the comprehensive write up, great work. I went on a small journey to try various workarounds but didn't get as far as yourself and others, particularly as my use-case is only a "nice-to-have". Most everything I found was in relation to Proxmox and others. Little did I know when I thought I was helping you over on Github, you would actually end up helping me!
As I understand from your notes, the full procedure would be:
Enable passthrough for the GPU on the host
Apply the patched wrapper to enable custom model args and respect platform vga
Disable emulated VGA via platform:vga=none
Apply the config to the VM via -device loader options
Reboot the host
Mount the GPU to the VM and boot it up