I'm having exactly the same problem with a "Tiger Jet Network Inc. / ICP DAS Tiger3XX Modem/ISDN interface" card.
logs show this:

May  5 16:13:17 xcp-ng xapi: [debug||412529 |Async.VM.start R:f406c721ea1c|xenops] xapi_cache: creating cache for ae56aad1-bab7-e6eb-8599-8eea93291e54
May  5 16:13:17 xcp-ng xapi: [debug||412529 |Async.VM.start R:f406c721ea1c|xenops] xenops_cache: creating empty cache for ae56aad1-bab7-e6eb-8599-8eea93291e54
May  5 16:13:17 xcp-ng xapi: [debug||412529 |Async.VM.start R:f406c721ea1c|xapi_network] register_vif vif=OpaqueRef:8b603d12-373e-4086-a03b-e63e45afb21e network=OpaqueRef:de3a7251-596a-4052-a90a-0caa23d93947
May  5 16:13:17 xcp-ng xapi: [ info||412529 |Async.VM.start R:f406c721ea1c|xenops] xenops: VM.start ae56aad1-bab7-e6eb-8599-8eea93291e54
May  5 16:13:17 xcp-ng xapi: [ info||412529 |Async.VM.start R:f406c721ea1c|xenops] xenops: Queueing VM.unpause ae56aad1-bab7-e6eb-8599-8eea93291e54
May  5 16:13:24 xcp-ng xapi: [ info||412529 |Async.VM.start R:f406c721ea1c|xapi_network] Caught Xenops_interface.Xenopsd_error([S(Internal_error);S(Cannot_add(0000:06:00.0, Xenctrlext.Unix_error(35, "3: No such process")))]): detaching networks
May  5 16:13:24 xcp-ng xapi: [error||412529 |Async.VM.start R:f406c721ea1c|xenops] Caught exception starting VM: Xenops_interface.Xenopsd_error([S(Internal_error);S(Cannot_add(0000:06:00.0, Xenctrlext.Unix_error(35, "3: No such process")))])
May  5 16:13:24 xcp-ng xapi: [debug||412529 |Async.VM.start R:f406c721ea1c|xenops] VM ae56aad1-bab7-e6eb-8599-8eea93291e54 set_resident_on
May  5 16:13:24 xcp-ng xapi: [debug||412529 |Async.VM.start R:f406c721ea1c|xenops] Signalling xenapi event thread to re-register, and xenopsd events to sync
May  5 16:13:24 xcp-ng xapi: [ info||412529 |Async.VM.start R:f406c721ea1c|xenops] xenops: UPDATES.refresh_vm ae56aad1-bab7-e6eb-8599-8eea93291e54
May  5 16:13:24 xcp-ng xapi: [debug||412529 |Async.VM.start R:f406c721ea1c|xenops] Client.UPDATES.inject_barrier 6
May  5 16:13:24 xcp-ng xapi: [debug||412529 |Async.VM.start R:f406c721ea1c|xenops] triggering xapi event thread to re-register via session.logout
May  5 16:13:24 xcp-ng xapi: [error||412529 |Async.VM.start R:f406c721ea1c|xenops] Re-raising as INTERNAL_ERROR [ xenopsd internal error: Cannot_add(0000:06:00.0, Xenctrlext.Unix_error(35, "3: No such process")) ]
May  5 16:13:24 xcp-ng xapi: [error||412529 ||backtrace] Async.VM.start R:f406c721ea1c failed with exception Server_error(INTERNAL_ERROR, [ xenopsd internal error: Cannot_add(0000:06:00.0, Xenctrlext.Unix_error(35, "3: No such process")) ])
May  5 16:13:24 xcp-ng xapi: [error||412529 ||backtrace] Raised Server_error(INTERNAL_ERROR, [ xenopsd internal error: Cannot_add(0000:06:00.0, Xenctrlext.Unix_error(35, "3: No such process")) ])
May  5 16:13:24 xcp-ng xapi: [error||412529 ||backtrace] 1/1 xapi Raised at file (Thread 412529 has no backtrace table. Was with_backtraces called?, line 0