Looks to have done the trick, no more 100's of tasks!
I also had an issue with backups whereby they were trying to access the XCP-NG host via their storage adaptors to which the appliance had to access to.
Your fix seems to have resolved that as well as I was able to rerun the backup after your fix was applied without issue.
I haven't successfully passed-through a graphics card with XCP-ng. The closest I have gotten was to pass-through the card and install the drivers, but it would not display anything. This was done with a previous version of XCP-ng, so that might be an avenue to take. I wasn't even able to install the drivers in the latest XCP-ng version.
I have, however, have had success with Qubes OS, so this might not apply 1 to 1.
Basically there is an issue with Xen which does not allow VMs with a graphics card passed-through to boot correctly when they are assigned more than 3.5GBs of ram. There is a work around for Qubes, which gets passed this.
You have to make sure that the graphics card is blacklisted at GRUB, so you shouldn't be able to see the XCP-ng console if you do it correctly, as you said.
Another problem that I've run into was installing the drivers. Basically, as soon as you are able after you've installed Windows, you should download your drivers, turn off the vm, and remove the network connection to the vm. Then, reboot the vm and install the drivers. Having the vm connected to the internet, for some reason would not allow the drivers to install correctly. The graphics card worked correctly after that.
I did run into a problem similar to yours when I passed-through my motherboard's usb's, however. With a 5600xt everything worked perfectly, but with a gtx 1070, the vm became unusable after passing-though the usb's, so this might be an issue with nvidia.
Since the issue is with Xen you might be able to duplicate the process above in XCP-ng.