@axel I have two graphics cards in the machine but I only pass one at a time to the VM. Either one can be seen by the guest, just not used for graphics as far as I can tell.

I have some experience with virtualization and GPU passthrough. When I've done it in a desktop rig it was with commercial GPUs and you'd connect a display to the GPU.

Question up front:
How do I "connect a display" to a passthrough datacenter GPU?

Description of steps taken:
I now have a second hand rig and I've unfortunately run up against the truth of Nvidia vGPUs. So before I shell out for a professional license I've decided to try to get XCP-ng to work in passthrough. I have an Ubuntu 18.04 server with GPU enabled Tensorflow/Keras. This was one goal of the server. The next is to setup a server based CAD machine. I've tried and I can get Debian/Ubuntu running with a GPU that nvidia-smi can enumerate. But it doesn't seem to be used for graphics acceleration. I generally access the machines through Xen Orchestra right now.

Rig:
Supermicro 2027GR-TRF w/ 2x E5-2680v2
64GB Ram and 4.5TB SSD array
Nvidia Tesla K20Xm and GRID K2
XCP-ng and XO
Guests: Debian family Linux machines