PCI Passthrough of QAT adapter IQA89601G1P5

[impovich]

Hello all, having an issue passing through QAT adapter to PFsense that I do not understand, so need some assistance.
On PFsense forum suggested that
"it does look like it's not passing that device through correctly. Or not completely."

c5:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
c6:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
c7:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)

[23.01-RELEASE][root@pfSense.home.lan]/root: kldstat -v | grep qat
11    1 0xffffffff8475d000     4348 qat.ko (/boot/kernel/qat.ko)
		695 nexus/qat
12    5 0xffffffff84762000    10e10 qat_hw.ko (/boot/kernel/qat_hw.ko)
		694 pci/qat_c4xxx
		690 pci/qat_c62x
		693 pci/qat_dh895xcc
		691 pci/qat_200xx
		692 pci/qat_c3xxx
13    7 0xffffffff84773000    29840 qat_common.ko (/boot/kernel/qat_common.ko)
		688 qat_common
14    6 0xffffffff8479d000    66b78 qat_api.ko (/boot/kernel/qat_api.ko)
		689 qat_api
15    1 0xffffffff84804000    11240 qat_c2xxx.ko (/boot/kernel/qat_c2xxx.ko)
		696 pci/qat_c2xxx

[23.01-RELEASE][root@pfSense.home.lan]/root: dmesg | grep qat
qat0: <Intel c6xx QuickAssist> mem 0xf1800000-0xf183ffff,0xf1840000-0xf187ffff irq 21 at device 9.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19
qat0: <Intel c6xx QuickAssist> mem 0xf1880000-0xf18bffff,0xf18c0000-0xf18fffff irq 25 at device 10.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19
qat0: <Intel c6xx QuickAssist> mem 0xf1900000-0xf193ffff,0xf1940000-0xf197ffff irq 29 at device 11.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19
qat0: <Intel c6xx QuickAssist> mem 0xf1800000-0xf183ffff,0xf1840000-0xf187ffff irq 21 at device 9.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19
qat0: <Intel c6xx QuickAssist> mem 0xf1880000-0xf18bffff,0xf18c0000-0xf18fffff irq 25 at device 10.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19
qat0: <Intel c6xx QuickAssist> mem 0xf1900000-0xf193ffff,0xf1940000-0xf197ffff irq 29 at device 11.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19
qat_ocf0: <QAT engine>
qat_ocf0: no QAT IRQ instances available
device_attach: qat_ocf0 attach returned 6
qat0: <Intel c6xx QuickAssist> mem 0xf1800000-0xf183ffff,0xf1840000-0xf187ffff irq 21 at device 9.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19
qat0: <Intel c6xx QuickAssist> mem 0xf1880000-0xf18bffff,0xf18c0000-0xf18fffff irq 25 at device 10.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19
qat0: <Intel c6xx QuickAssist> mem 0xf1900000-0xf193ffff,0xf1940000-0xf197ffff irq 29 at device 11.0 on pci0
qat0: Unable to find AER capability of the device
device_attach: qat0 attach returned -19

[21:44 dallas ~]# dmesg | grep pciback
[    0.000000] Command line: root=LABEL=root-bhmxgw ro nolvm hpet=disable console=hvc0 console=tty0 quiet vga=785 splash plymouth.ignore-serial-consoles xen-pciback.hide=(0000:c5:00.0)(0000:c6:00.0)(0000:c7:00.0)
[    1.811384] Kernel command line: root=LABEL=root-bhmxgw ro nolvm hpet=disable console=hvc0 console=tty0 quiet vga=785 splash plymouth.ignore-serial-consoles xen-pciback.hide=(0000:c5:00.0)(0000:c6:00.0)(0000:c7:00.0)
[    4.419889] pciback 0000:c5:00.0: seizing device
[    4.419895] pciback 0000:c6:00.0: seizing device
[    4.419900] pciback 0000:c7:00.0: seizing device
[    4.771730] pciback 0000:c7:00.0: enabling device (0140 -> 0142)
[    4.771804] pciback 0000:c7:00.0: %(#e41111)[Disabling System Error reporting on root port c2:04.0]
[    4.876894] pciback 0000:c6:00.0: enabling device (0140 -> 0142)
[    4.984902] pciback 0000:c5:00.0: enabling device (0140 -> 0142)
[    5.092811] xen_pciback: backend is vpci

c5:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
	Subsystem: Intel Corporation QuickAssist Adapter 8960
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 16
	Region 2: Memory at fb540000 (64-bit, non-prefetchable) [size=256K]
	Region 4: Memory at fb500000 (64-bit, non-prefetchable) [size=256K]
	Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [60] MSI-X: Enable- Count=17 Masked-
		Vector table: BAR=2 offset=0003b000
		PBA: BAR=2 offset=0003b800
	Capabilities: [6c] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [74] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <128ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [138 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [140 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function Dependency Link: 00
		VF offset: 8, stride: 1, Device ID: 37c9
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000fb590000 (64-bit, non-prefetchable)
		Region 2: Memory at 00000000fb580000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Capabilities: [1b0 v1] Access Control Services
		ACSCap:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
	Kernel driver in use: pciback

c6:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
	Subsystem: Intel Corporation QuickAssist Adapter 8960
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 17
	Region 2: Memory at fb440000 (64-bit, non-prefetchable) [size=256K]
	Region 4: Memory at fb400000 (64-bit, non-prefetchable) [size=256K]
	Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [60] MSI-X: Enable- Count=17 Masked-
		Vector table: BAR=2 offset=0003b000
		PBA: BAR=2 offset=0003b800
	Capabilities: [6c] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [74] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <128ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [138 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [140 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function Dependency Link: 00
		VF offset: 8, stride: 1, Device ID: 37c9
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000fb490000 (64-bit, non-prefetchable)
		Region 2: Memory at 00000000fb480000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Capabilities: [1b0 v1] Access Control Services
		ACSCap:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
	Kernel driver in use: pciback

c7:00.0 Co-processor: Intel Corporation C62x Chipset QuickAssist Technology (rev 04)
	Subsystem: Intel Corporation QuickAssist Adapter 8960
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 32 bytes
	Interrupt: pin A routed to IRQ 18
	Region 2: Memory at fb340000 (64-bit, non-prefetchable) [size=256K]
	Region 4: Memory at fb300000 (64-bit, non-prefetchable) [size=256K]
	Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [60] MSI-X: Enable- Count=17 Masked-
		Vector table: BAR=2 offset=0003b000
		PBA: BAR=2 offset=0003b800
	Capabilities: [6c] Power Management version 3
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [74] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <128ns, L1 <1us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x16, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
		UESvrt:	DLP+ SDES- TLP+ FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [138 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [140 v1] Single Root I/O Virtualization (SR-IOV)
		IOVCap:	Migration-, Interrupt Message Number: 000
		IOVCtl:	Enable- Migration- Interrupt- MSE- ARIHierarchy+
		IOVSta:	Migration-
		Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function Dependency Link: 00
		VF offset: 8, stride: 1, Device ID: 37c9
		Supported Page Size: 00000553, System Page Size: 00000001
		Region 0: Memory at 00000000fb390000 (64-bit, non-prefetchable)
		Region 2: Memory at 00000000fb380000 (64-bit, non-prefetchable)
		VF Migration: offset: 00000000, BIR: 0
	Capabilities: [1b0 v1] Access Control Services
		ACSCap:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
	Kernel driver in use: pciback

Also the card is visible via xl pci-assignable-list after i passed it to pfsense. not sure if it is expected or not

[sapcode]

@impovich I have the same issue with Intel Atom Processor c3758, in XCP 8.2.1 the co-processor is visible:

lspci -nn
01:00.0 Co-processor [0b40]: Intel Corporation Atom Processor C3000 Series QuickAssist Technology [8086:19e2] (rev 11)
 
lspci -vd 8086:19e2
01:00.0 Co-processor: Intel Corporation Atom Processor C3000 Series QuickAssist Technology (rev 11)
        Subsystem: Intel Corporation Device 0000
        Flags: fast devsel, IRQ 255
        Memory at df440000 (64-bit, non-prefetchable) [disabled] [size=256K]
        Memory at df400000 (64-bit, non-prefetchable) [disabled] [size=256K]
        Capabilities: [b0] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [60] MSI-X: Enable- Count=17 Masked-
        Capabilities: [6c] Power Management version 3
        Capabilities: [74] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [138] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [140] Single Root I/O Virtualization (SR-IOV)
        Capabilities: [1b0] Access Control Services

But i am missing those entrie:

 Kernel driver in use: c3xxx  <- Not present
 Kernel modules: qat_c3xxx  <- Not present

Also "xl pci-assignable-list" show's ntothing.

In the OpNsense VM i get this, without any QAT co-processor:

lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation 440FX - 82441FX PMC [Natoma] [8086:1237] (rev 02)
00:01.0 ISA bridge [0601]: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II] [8086:7000]
00:01.1 IDE interface [0101]: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] [8086:7010]
00:01.2 USB controller [0c03]: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] [8086:7020] (rev 01)
00:01.3 Bridge [0680]: Intel Corporation 82371AB/EB/MB PIIX4 ACPI [8086:7113] (rev 01)
00:02.0 VGA compatible controller [0300]: Cirrus Logic GD 5446 [1013:00b8]
00:03.0 SCSI storage controller [0100]: XenSource, Inc. Xen Platform Device [5853:0001] (rev 01)

Is there anything which can be done to pass it through ?

@olivierlambert I remeber there where some threads about GPU pass through with some crazy hacks maybe we could try appling some stuff here ?

Best regards

[impovich]

@sapcode Sorry to hear you're experiencing the same issue. I couldn't find a solution, and QAT was sold long ago—no QAT, no issues.

[Teddy Astie]

PCIe AER needs proper PCIe, which in practice needs Q35 chipset in the guest (or some other guest type/PCI passthrough way).

Q35 support is currently work in progress

[olivierlambert]

Yes it's a work in progress and @ThierryEscande is working on it.

edit: @Teddy-Astie another point to add in our discussion for the next Xen Summit.

[sapcode]

@olivierlambert thank you for the quick answers, just to put it into context the plan was to use accelerated Wireguard VPN in pfSense+ which comes with own FreeBSD optimized kernel modules:

Some more info on CPU extensions / QAT / IPSsec-MB / AES-NI related to that:

1. https://github.com/intel/intel-ipsec-mb
2. https://docs.netgate.com/pfsense/en/latest/hardware/cryptographic-accelerators.html
3. https://networkbuilders.intel.com/docs/networkbuilders/intel-avx-512-and-intel-qat-accelerate-wireguard-processing-with-intel-xeon-d-2700-processor-technology-guide-1647024663.pdf
4. https://2023.asiabsdcon.org/program/_p05b/paper.pdf
5. https://forum.opnsense.org/index.php?topic=37808.0
6. https://forum.netgate.com/topic/173606/solved-intel-qat-driver-with-wireguard-support/9
7. https://github.com/pfsense/FreeBSD-src/blob/devel-main/sys/dev/qat/include/common/adf_accel_devices.h#L38

Example CPU's having the relevant features incl. AVX-512:

1. https://ark.intel.com/content/www/de/de/ark/products/226102/intel-xeon-d-1713nt-processor-10m-cache-up-to-3-50-ghz.html
2. https://ark.intel.com/content/www/de/de/ark/products/226113/intel-xeon-d-1722ne-processor-10m-cache-up-to-2-70-ghz.html

Some older Atoms have Intel SSE4.2 but miss the AVX-512 which i think bring the most performance:

1. https://ark.intel.com/content/www/de/de/ark/products/97937/intel-atom-processor-c3558-8m-cache-up-to-2-20-ghz.html
2. https://ark.intel.com/content/www/de/de/ark/products/134803/intel-atom-processor-c5115-9m-cache-2-8ghz.html

Best regards