@TheFrisianClause
From my experiments using a GTX1080ti you'll have to follow the instructions for generic pcie-passthrough to the letter, which mostly means that the passthrough needs to be done on both sides, on the Dom0 for relinquishing device control and for the DomU to pick up the device (section 5). Perhaps now that the restrictions from Nvidia have gone, Vates will kindly include some GUI support for those operations in XOA.
Note, that if your dGPU has multiple devices (e.g. my GTX 1080ti also has a USB-C controller on-board), both entries need to be added in a single 'xe vm-param-set' statement, otherwise only the latter device (USB in my case) will wind up in the VM.... (yeah, at least 30 minutes of puzzling on that one)
Of course, if the dGPU is your console device, it means flying blind afterwards, but I'm getting used to that with all the recent iGPUs as well (then again, I have some DisplayLink hardware that's currently unused and EL7/8 drivers for those have popped up recently...)
Thankfully the dreaded error 43 issues have gone away with the more recent Nvidia drivers, sadly Kepler support has been retired (got a lot of those still around), so you may want to preserve the latest CUDA 11.4 release as one that offers both, but for Maxwell you should still be fine.
Before trying to diagnose with the Nvidia drivers, you should be able to see the device transition via lspci on both sides, Dom0 and DomU.