pci passthru issue

[kahjs2007]

Trying to pass two storage controllers into a new vm after deleting the vm that used them before

I have verified the hardware is still there:

18:39 xen2 ~]# /opt/xensource/libexec/xen-cmdline --delete-dom0 xen-pciback.hide
[18:40 xen2 ~]# reboot

[18:44 xen2 ~]# lsblk
NAME                                                                                              MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sdf                                                                                                 8:80   0 931.5G  0 disk 
nvme0n1                                                                                           259:0    0   477G  0 disk 
├─nvme0n1p5                                                                                       259:5    0     4G  0 part /var/log
├─nvme0n1p3                                                                                       259:3    0 435.4G  0 part 
│ └─XSLocalEXT--6570b794--3efe--80fe--d273--8ccf1e7d1ec3-6570b794--3efe--80fe--d273--8ccf1e7d1ec3 253:0    0 435.4G  0 lvm  /run/sr-mount/6570b794-3efe-80fe-d273-8ccf1e7d1ec3
├─nvme0n1p1                                                                                       259:1    0    18G  0 part /
├─nvme0n1p6                                                                                       259:6    0     1G  0 part [SWAP]
├─nvme0n1p4                                                                                       259:4    0   512M  0 part /boot/efi
└─nvme0n1p2                                                                                       259:2    0    18G  0 part 
sdd                                                                                                 8:48   0 931.5G  0 disk 
sdb                                                                                                 8:16   0 931.5G  0 disk 
sdk                                                                                                 8:160  0 223.6G  0 disk 
sdi                                                                                                 8:128  0 238.5G  0 disk 
sdg                                                                                                 8:96   0 931.5G  0 disk 
sde                                                                                                 8:64   0 931.5G  0 disk 
sdc                                                                                                 8:32   0 931.5G  0 disk 
sda                                                                                                 8:0    0 931.5G  0 disk 
sdj                                                                                                 8:144  0 223.6G  0 disk 
sdh                                                                                                 8:112  0 931.5G  0 disk 
[18:44 xen2 ~]# lspci | egrep -i "sata|sas"
00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)
04:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05)
[18:44 xen2 ~]# /opt/xensource/libexec/xen-cmdline --set-dom0 "xen-pciback.hide=(0000:00:1f.2)(0000:04:00.0)"
[18:46 xen2 ~]# reboot

[18:49 xen2 ~]# xl pci-assignable-list 
0000:00:1f.2
0000:04:00.0
[18:49 xen2 ~]# xe vm-param-set other-config:pci=0/0000:00:1f.2,0/0000:04:00.0 uuid=0bbfdac9-8ec0-938b-ccdb-0cfe7927e6c7

getting this error when trying to start the vm

vm.start
{
  "id": "0bbfdac9-8ec0-938b-ccdb-0cfe7927e6c7",
  "bypassMacAddressesCheck": false,
  "force": false
}
{
  "code": "INTERNAL_ERROR",
  "params": [
    "xenopsd internal error: Cannot_add(0000:04:00.0, Device_common.QMP_Error(138, \"{\\\"error\\\":{\\\"class\\\":\\\"GenericError\\\",\\\"desc\\\":\\\"Failed to initialize 11/15, type = 0x1, rc: -1\\\",\\\"data\\\":{}},\\\"id\\\":\\\"qmp-001026-138\\\"}\"))"
  ],
  "call": {
    "method": "VM.start",
    "params": [
      "OpaqueRef:90abe1a0-8fa2-4913-a01b-6933fb75d442",
      false,
      false
    ]
  },
  "message": "INTERNAL_ERROR(xenopsd internal error: Cannot_add(0000:04:00.0, Device_common.QMP_Error(138, \"{\\\"error\\\":{\\\"class\\\":\\\"GenericError\\\",\\\"desc\\\":\\\"Failed to initialize 11/15, type = 0x1, rc: -1\\\",\\\"data\\\":{}},\\\"id\\\":\\\"qmp-001026-138\\\"}\")))",
  "name": "XapiError",
  "stack": "XapiError: INTERNAL_ERROR(xenopsd internal error: Cannot_add(0000:04:00.0, Device_common.QMP_Error(138, \"{\\\"error\\\":{\\\"class\\\":\\\"GenericError\\\",\\\"desc\\\":\\\"Failed to initialize 11/15, type = 0x1, rc: -1\\\",\\\"data\\\":{}},\\\"id\\\":\\\"qmp-001026-138\\\"}\")))
    at Function.wrap (file:///opt/xo/xo-builds/xen-orchestra-202409101827/packages/xen-api/_XapiError.mjs:16:12)
    at file:///opt/xo/xo-builds/xen-orchestra-202409101827/packages/xen-api/transports/json-rpc.mjs:38:21
    at runNextTicks (node:internal/process/task_queues:60:5)
    at processImmediate (node:internal/timers:454:9)
    at process.callbackTrampoline (node:internal/async_hooks:130:17)"
}

any thoughts ?

This machine is a pool master, so contemplating moving the master to another machine and reinstalling this one.

[kahjs2007]

Ok so I'm going to answer my own question.

The answer is that the vdi (local SR) you are trying to boot from and the pci slots are on two different machines and you need to migrate the boot vdi to a shared pool sr or to the local sr of the machine with the pci slots!