@sapcode Sorry to hear you're experiencing the same issue. I couldn't find a solution, and QAT was sold long ago—no QAT, no issues. 
I
Posts made by impovich
-
RE: PCI Passthrough of QAT adapter IQA89601G1P5posted in Compute
-
PCI Passthrough of QAT adapter IQA89601G1P5posted in Compute
Hello all, having an issue passing through QAT adapter to PFsense that I do not understand, so need some assistance.
On PFsense forum suggested that
"it does look like it's not passing that device through correctly. Or not completely."c5:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04) c6:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04) c7:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)[23.01-RELEASE][root@pfSense.home.lan]/root: kldstat -v | grep qat 11 1 0xffffffff8475d000 4348 qat.ko (/boot/kernel/qat.ko) 695 nexus/qat 12 5 0xffffffff84762000 10e10 qat_hw.ko (/boot/kernel/qat_hw.ko) 694 pci/qat_c4xxx 690 pci/qat_c62x 693 pci/qat_dh895xcc 691 pci/qat_200xx 692 pci/qat_c3xxx 13 7 0xffffffff84773000 29840 qat_common.ko (/boot/kernel/qat_common.ko) 688 qat_common 14 6 0xffffffff8479d000 66b78 qat_api.ko (/boot/kernel/qat_api.ko) 689 qat_api 15 1 0xffffffff84804000 11240 qat_c2xxx.ko (/boot/kernel/qat_c2xxx.ko) 696 pci/qat_c2xxx[23.01-RELEASE][root@pfSense.home.lan]/root: dmesg | grep qat qat0: <Intel c6xx QuickAssist> mem 0xf1800000-0xf183ffff,0xf1840000-0xf187ffff irq 21 at device 9.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19 qat0: <Intel c6xx QuickAssist> mem 0xf1880000-0xf18bffff,0xf18c0000-0xf18fffff irq 25 at device 10.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19 qat0: <Intel c6xx QuickAssist> mem 0xf1900000-0xf193ffff,0xf1940000-0xf197ffff irq 29 at device 11.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19 qat0: <Intel c6xx QuickAssist> mem 0xf1800000-0xf183ffff,0xf1840000-0xf187ffff irq 21 at device 9.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19 qat0: <Intel c6xx QuickAssist> mem 0xf1880000-0xf18bffff,0xf18c0000-0xf18fffff irq 25 at device 10.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19 qat0: <Intel c6xx QuickAssist> mem 0xf1900000-0xf193ffff,0xf1940000-0xf197ffff irq 29 at device 11.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19 qat_ocf0: <QAT engine> qat_ocf0: no QAT IRQ instances available device_attach: qat_ocf0 attach returned 6 qat0: <Intel c6xx QuickAssist> mem 0xf1800000-0xf183ffff,0xf1840000-0xf187ffff irq 21 at device 9.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19 qat0: <Intel c6xx QuickAssist> mem 0xf1880000-0xf18bffff,0xf18c0000-0xf18fffff irq 25 at device 10.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19 qat0: <Intel c6xx QuickAssist> mem 0xf1900000-0xf193ffff,0xf1940000-0xf197ffff irq 29 at device 11.0 on pci0 qat0: Unable to find AER capability of the device device_attach: qat0 attach returned -19[21:44 dallas ~]# dmesg | grep pciback [ 0.000000] Command line: root=LABEL=root-bhmxgw ro nolvm hpet=disable console=hvc0 console=tty0 quiet vga=785 splash plymouth.ignore-serial-consoles xen-pciback.hide=(0000:c5:00.0)(0000:c6:00.0)(0000:c7:00.0) [ 1.811384] Kernel command line: root=LABEL=root-bhmxgw ro nolvm hpet=disable console=hvc0 console=tty0 quiet vga=785 splash plymouth.ignore-serial-consoles xen-pciback.hide=(0000:c5:00.0)(0000:c6:00.0)(0000:c7:00.0) [ 4.419889] pciback 0000:c5:00.0: seizing device [ 4.419895] pciback 0000:c6:00.0: seizing device [ 4.419900] pciback 0000:c7:00.0: seizing device [ 4.771730] pciback 0000:c7:00.0: enabling device (0140 -> 0142) [ 4.771804] pciback 0000:c7:00.0: %(#e41111)[Disabling System Error reporting on root port c2:04.0] [ 4.876894] pciback 0000:c6:00.0: enabling device (0140 -> 0142) [ 4.984902] pciback 0000:c5:00.0: enabling device (0140 -> 0142) [ 5.092811] xen_pciback: backend is vpcic5:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04) Subsystem: Intel Corporation QuickAssist Adapter 8960 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Interrupt: pin A routed to IRQ 16 Region 2: Memory at fb540000 (64-bit, non-prefetchable) [size=256K] Region 4: Memory at fb500000 (64-bit, non-prefetchable) [size=256K] Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [60] MSI-X: Enable- Count=17 Masked- Vector table: BAR=2 offset=0003b000 PBA: BAR=2 offset=0003b800 Capabilities: [6c] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [74] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <128ns, L1 <1us ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR-, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UESvrt: DLP+ SDES- TLP+ FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [138 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [140 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+ IOVSta: Migration- Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function Dependency Link: 00 VF offset: 8, stride: 1, Device ID: 37c9 Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000fb590000 (64-bit, non-prefetchable) Region 2: Memory at 00000000fb580000 (64-bit, non-prefetchable) VF Migration: offset: 00000000, BIR: 0 Capabilities: [1b0 v1] Access Control Services ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- Kernel driver in use: pciback c6:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04) Subsystem: Intel Corporation QuickAssist Adapter 8960 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Interrupt: pin A routed to IRQ 17 Region 2: Memory at fb440000 (64-bit, non-prefetchable) [size=256K] Region 4: Memory at fb400000 (64-bit, non-prefetchable) [size=256K] Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [60] MSI-X: Enable- Count=17 Masked- Vector table: BAR=2 offset=0003b000 PBA: BAR=2 offset=0003b800 Capabilities: [6c] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [74] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <128ns, L1 <1us ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR-, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UESvrt: DLP+ SDES- TLP+ FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [138 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [140 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+ IOVSta: Migration- Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function Dependency Link: 00 VF offset: 8, stride: 1, Device ID: 37c9 Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000fb490000 (64-bit, non-prefetchable) Region 2: Memory at 00000000fb480000 (64-bit, non-prefetchable) VF Migration: offset: 00000000, BIR: 0 Capabilities: [1b0 v1] Access Control Services ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- Kernel driver in use: pciback c7:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04) Subsystem: Intel Corporation QuickAssist Adapter 8960 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 32 bytes Interrupt: pin A routed to IRQ 18 Region 2: Memory at fb340000 (64-bit, non-prefetchable) [size=256K] Region 4: Memory at fb300000 (64-bit, non-prefetchable) [size=256K] Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [60] MSI-X: Enable- Count=17 Masked- Vector table: BAR=2 offset=0003b000 PBA: BAR=2 offset=0003b800 Capabilities: [6c] Power Management version 3 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [74] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <128ns, L1 <1us ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset- MaxPayload 256 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR-, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol- UESvrt: DLP+ SDES- TLP+ FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [138 v1] Alternative Routing-ID Interpretation (ARI) ARICap: MFVC- ACS-, Next Function: 0 ARICtl: MFVC- ACS-, Function Group: 0 Capabilities: [140 v1] Single Root I/O Virtualization (SR-IOV) IOVCap: Migration-, Interrupt Message Number: 000 IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+ IOVSta: Migration- Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function Dependency Link: 00 VF offset: 8, stride: 1, Device ID: 37c9 Supported Page Size: 00000553, System Page Size: 00000001 Region 0: Memory at 00000000fb390000 (64-bit, non-prefetchable) Region 2: Memory at 00000000fb380000 (64-bit, non-prefetchable) VF Migration: offset: 00000000, BIR: 0 Capabilities: [1b0 v1] Access Control Services ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans- Kernel driver in use: pcibackAlso the card is visible via
xl pci-assignable-listafter i passed it to pfsense. not sure if it is expected or not -
RE: Migrating VM fails with DUPLICATE_VM errorposted in Xen Orchestra
@olivierlambert having exactly the same issue as wcom939 reported.
What did you mean by "Can you try with xe command to see if you have the same error?" ? Migrate using CLI? there are too many options:) -
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@jmccoy555 got it. Thank you for being with me during this amazing journey
-
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@jmccoy555 sorry but didn't get it. I'm a newbie
-
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
Finally, I found a solution to the issue:
https://www.truenas.com/community/threads/multiple-ada0-partitions-on-xen-pv-w-passthrough.16574/set via GUI tunables interface:
hint.ada.0.at="scbus100".Works perfectly!
root@truenas[~]# geom disk list Geom name: ada0 Providers: 1. Name: ada0 Mediasize: 53687091200 (50G) Sectorsize: 512 Mode: r1w1e2 descr: (null) ident: (null) rotationrate: unknown fwsectors: 0 fwheads: 0 Geom name: cd0 Providers: 1. Name: cd0 Mediasize: 0 (0B) Sectorsize: 2048 Mode: r0w0e0 descr: QEMU QEMU DVD-ROM ident: (null) rotationrate: unknown fwsectors: 0 fwheads: 0 Geom name: ada1 Providers: 1. Name: ada1 Mediasize: 4000787030016 (3.6T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r0w0e0 descr: ST4000LM024-2U817V lunid: 5000c500d5b4af0c ident: WCK7ABLM rotationrate: 5400 fwsectors: 63 fwheads: 16 Geom name: ada2 Providers: 1. Name: ada2 Mediasize: 4000787030016 (3.6T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r0w0e0 descr: ST4000LM024-2U817V lunid: 5000c500d5b3d1cb ident: WCK7ADS1 rotationrate: 5400 fwsectors: 63 fwheads: 16 -
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@jmccoy555 so managed to do it with a dirty hack
- Spwaned a new VM with 3 virtual drives
ada0 - 5gb
ada1 - 5gb
ada2 - 50gb - installed Truenas on ada2.
- After installation removed ada0 and ada1
- Rebooted VM and confirmed that ada2 is still ada2
- passed through SATA controller
- Bingo!!
root@truenas[~]# geom disk list Geom name: ada2 Providers: 1. Name: ada2 Mediasize: 53687091200 (50G) Sectorsize: 512 Mode: r1w1e2 descr: (null) Geom name: cd0 Providers: 1. Name: cd0 Mediasize: 0 (0B) Sectorsize: 2048 Mode: r0w0e0 descr: QEMU QEMU DVD-ROM ident: (null) Geom name: ada0 Providers: 1. Name: ada0 Mediasize: 4000787030016 (3.6T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r0w0e0 descr: ST4000LM024-2U817V lunid: 5000c500d5 fwheads: 16 Geom name: ada1 Providers: 1. Name: ada1 Mediasize: 4000787030016 (3.6T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r0w0e0 - Spwaned a new VM with 3 virtual drives
-
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@jmccoy555 said in Sata controller passthrough. VM no bootable device:
@impovich Ah yeah. Only passed though a controller to Linux VMs and that works fine. Could try TrueNAS Scale??
https://forums.freebsd.org/threads/dev-ad4-becomes-dev-ada0.29899/ Any help?
I'm out of ideas I'm afraid.
I tried to do the same with Debian, no success. Reading freebsd forum regarding labeling
-
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@jmccoy555 i guess you have an HBA controller?
-
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@jmccoy555 i just tried what you've suggested, unfortunately two ada0 disks again
root@truenas[~]# geom disk list Geom name: ada0 Providers: 1. Name: ada0 Mediasize: 53687091200 (50G) Sectorsize: 512 Mode: r1w1e2 descr: (null) ident: (null) rotationrate: unknown fwsectors: 0 fwheads: 0 Geom name: cd0 Providers: 1. Name: cd0 Mediasize: 0 (0B) Sectorsize: 2048 Mode: r0w0e0 descr: QEMU QEMU DVD-ROM ident: (null) rotationrate: unknown fwsectors: 0 fwheads: 0 Geom name: ada0 Providers: 1. Name: ada0 Mediasize: 4000787030016 (3.6T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r1w1e3 descr: ST4000LM024-2U817V lunid: 5000c500d5b4af0c ident: WCK7ABLM rotationrate: 5400 fwsectors: 63 fwheads: 16 Geom name: ada1 Providers: 1. Name: ada1 Mediasize: 4000787030016 (3.6T) Sectorsize: 512 Stripesize: 4096 Stripeoffset: 0 Mode: r2w2e5 descr: ST4000LM024-2U817V lunid: 5000c500d5b3d1cb ident: WCK7ADS1 rotationrate: 5400 fwsectors: 63 fwheads: 16 -
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@jmccoy555 sorry, I was too quick and didn't read your message properly. (sorry)
Truenas installed on virtual HDD, physical HDD drives are passed though individually using symlinks. -
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@jmccoy555 Actually that is simple to do, it is already up and running
I just wanted to pass through the whole controller in order to have SMART checks and so on, as for snapshots - it is not possible to make them until everything is under xcp-ng itself.EDIT:
get TrueNAS up and running on a virtual disk, shut down, pass through the controller, boot, create your pool from your passed thought disks, and off you go.hm, that might work actually, have not tried it this way. Currently, I passed through separate HDD drives.
-
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
@olivierlambert just tried to install Debian using the same approach, got exactly the same issue.
-
RE: Sata controller passthrough. VM no bootable deviceposted in Compute
I tried to install Truenas on virtual HDD which is M.2 SSD and passthrough the SATA controller, ended up having two ada0 devices.
-
Sata controller passthrough. VM no bootable deviceposted in Compute
Hi all, trying to install Truenas Core 12 on xcp-ng 8.2.
Passed through SATA controller to a VM, installer detected hard drives, installed Truenas and after reboot, I'm getting "no bootable device".
What I did:[10:56 chicago ~]# lspci 00:00.0 Host bridge: Intel Corporation Device 3e33 (rev 07) 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (rev 07) 00:08.0 System peripheral: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th Gen Core Processor Gaussian Mixture Model 00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10) 00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10) 00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10) 00:15.0 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH Serial IO I2C Controller #0 (rev 10) 00:15.1 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH Serial IO I2C Controller #1 (rev 10) 00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10) 00:16.4 Communication controller: Intel Corporation Device a364 (rev 10) 00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10) 00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #22 (rev f0) 00:1b.6 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #23 (rev f0) 00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0) 00:1c.1 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #2 (rev f0) 00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0) 00:1e.0 Communication controller: Intel Corporation Device a328 (rev 10) 00:1f.0 ISA bridge: Intel Corporation Device a30a (rev 10) 00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10) 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller (rev 10) 01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01) 02:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) 03:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03) 05:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 04) 06:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 41) 07:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/opt/xensource/libexec/xen-cmdline --set-dom0 "xen-pciback.hide=(00:17.0)"[11:08 chicago ~]# xe vm-list uuid ( RO) : 7d16b32f-fe80-4271-8543-68c5a642220e name-label ( RW): Control domain on host: chicago power-state ( RO): running uuid ( RO) : adfa344c-d136-885f-8396-4f332ad40770 name-label ( RW): Truenas power-state ( RO): halted[11:25 chicago ~]# xl pci-assignable-list 0000:00:17.0xe vm-param-set other-config:pci=0/0000:00:17.0 uuid=adfa344c-d136-885f-8396-4f332ad40770[11:27 chicago ~]# xe host-dmesg | grep VT-d (XEN) [ 2.053436] Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB (XEN) [ 2.066660] Intel VT-d Snoop Control enabled. (XEN) [ 2.079753] Intel VT-d Dom0 DMA Passthrough not enabled. (XEN) [ 2.092849] Intel VT-d Queued Invalidation enabled. (XEN) [ 2.105873] Intel VT-d Interrupt Remapping enabled. (XEN) [ 2.118799] Intel VT-d Posted Interrupt not enabled. (XEN) [ 2.131762] Intel VT-d Shared EPT tables enabled.What am I doing wrong?
EDIT:
I didn't add virtual HDD, want to install on the real HDD