Guest receiving passthrough SATA controllers does not see attached drives
-
Re: Guest receiving passthrough SATA controllers does not see attached drives
It seems i'm having the same issue, using an AMD Epyc 7313P with Asrock Rack ROMED6U-2L2T motherboard and also having problem with passthrough the Advanced Micro Devices, Inc. [AMD] FCH SATA Controllers.
*01:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0]) Subsystem: ZyDAS Technology Corp. ASM1166 Serial ATA Controller Flags: bus master, fast devsel, latency 0, IRQ 132, IOMMU group 74 Memory at f8982000 (32-bit, non-prefetchable) [size=8K] Memory at f8980000 (32-bit, non-prefetchable) [size=8K] Expansion ROM at f8900000 [disabled] [size=512K] Capabilities: [40] Power Management version 3 Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+ Capabilities: [80] Express Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Capabilities: [130] Secondary PCI Express Kernel driver in use: ahci Kernel modules: ahci 45:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0]) Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] Flags: bus master, fast devsel, latency 0, IRQ 74, IOMMU group 59 Memory at b2400000 (32-bit, non-prefetchable) [size=2K] Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Capabilities: [64] Express Endpoint, MSI 00 Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+ Capabilities: [d0] SATA HBA v1.0 Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150] Advanced Error Reporting Capabilities: [270] Secondary PCI Express Capabilities: [2a0] Access Control Services Capabilities: [370] Transaction Processing Hints Capabilities: [400] Data Link Feature <?> Capabilities: [410] Physical Layer 16.0 GT/s <?> Capabilities: [440] Lane Margining at the Receiver <?> Kernel driver in use: ahci Kernel modules: ahci 46:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0]) Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] Flags: bus master, fast devsel, latency 0, IRQ 99, IOMMU group 60 Memory at b2300000 (32-bit, non-prefetchable) [size=2K] Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Capabilities: [64] Express Endpoint, MSI 00 Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+ Capabilities: [d0] SATA HBA v1.0 Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150] Advanced Error Reporting Capabilities: [270] Secondary PCI Express Capabilities: [2a0] Access Control Services Capabilities: [370] Transaction Processing Hints Capabilities: [400] Data Link Feature <?> Capabilities: [410] Physical Layer 16.0 GT/s <?> Capabilities: [440] Lane Margining at the Receiver <?> Kernel driver in use: ahci Kernel modules: ahci* *45:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0]) Subsystem: Advanced Micro Devices, Inc. [AMD] Device 7901 Flags: bus master, fast devsel, latency 0, IRQ 105 Memory at b2400000 (32-bit, non-prefetchable) Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Capabilities: [64] Express Endpoint, MSI 00 Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+ Capabilities: [d0] SATA HBA v1.0 Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150] Advanced Error Reporting Capabilities: [270] Secondary PCI Express Capabilities: [2a0] Access Control Services Capabilities: [370] Transaction Processing Hints Capabilities: [400] Data Link Feature <?> Capabilities: [410] Physical Layer 16.0 GT/s <?> Capabilities: [440] Lane Margining at the Receiver <?> 46:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0]) Subsystem: Advanced Micro Devices, Inc. [AMD] Device 7901 Flags: bus master, fast devsel, latency 0, IRQ 109 Memory at b2300000 (32-bit, non-prefetchable) Capabilities: [48] Vendor Specific Information: Len=08 <?> Capabilities: [50] Power Management version 3 Capabilities: [64] Express Endpoint, MSI 00 Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+ Capabilities: [d0] SATA HBA v1.0 Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?> Capabilities: [150] Advanced Error Reporting Capabilities: [270] Secondary PCI Express Capabilities: [2a0] Access Control Services Capabilities: [370] Transaction Processing Hints Capabilities: [400] Data Link Feature <?> Capabilities: [410] Physical Layer 16.0 GT/s <?> Capabilities: [440] Lane Margining at the Receiver <?>*
When I passthrough the PCIe ASM1166 card, can see and use the drives (that are connected to this card) within the VM.
But when I passthrough the Advanced Micro Devices, Inc. [AMD] FCH SATA Controllers [AHCI mode], don't see any connected drives to these controllers.
In the guest os (tested with Ubuntu and Truenas Core & Scale) i'm getting drive error.For example, this are a few errors i get in Truenas Core:
*root@truenas[~]# camcontrol reportluns 0:0:0 (pass0:ahcich0:0:0:0): REPORT LUNS. CDB: a0 00 00 00 00 00 00 00 00 10 00 00 (pass0:ahcich0:0:0:0): CAM status: CCB request was invalid root@truenas[~]# camcontrol epc ada0 -c list (pass0:ahcich0:0:0:0): READ_LOG_DMA_EXT. ACB: 47 00 08 00 00 40 00 00 00 00 02 00 (pass0:ahcich0:0:0:0): CAM status: ATA Status Error (pass0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) (pass0:ahcich0:0:0:0): RES: 51 04 08 00 00 40 00 00 00 02 00 root@truenas[~]# pciconf -lv ahci2@pci0:1:0:0: class=0x010601 rev=0x02 hdr=0x00 vendor=0x1b21 device=0x1166 subvendor=0x2116 subdevice=0x2116 vendor = 'ASMedia Technology Inc.' class = mass storage subclass = SATA ahci0@pci0:69:0:0: class=0x010601 rev=0x51 hdr=0x00 vendor=0x1022 device=0x7904 subvendor=0x1022 subdevice=0x7901 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'FCH SATA Controller [AHCI mode]' class = mass storage subclass = SATA ahci1@pci0:70:0:0: class=0x010601 rev=0x51 hdr=0x00 vendor=0x1022 device=0x7904 subvendor=0x1022 subdevice=0x7901 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'FCH SATA Controller [AHCI mode]' class = mass storage subclass = SATA*
When passthrough is disabled, i can see and use the connected disks just fine within XCP-NG and then no errors are reported.
Any thoughts?
-
Hello @hvm,
Can you give the output of
xl dmesg
in XCP-ng and ofdmesg
in the guest that has the issues ?
I have the impression that something is going wrong with reserved regions related to the SATA controller. -
Hi @Teddy-Astie
First of all, thanks for your thoughts!
I'm pretty stuck with these issues. So hopefully other insights will help
Below are the requested log files.xl_dmesg_passthrough_enabled-XCPNG_host.txt
pci_devices-XCPNG_host.txt
dmesg_VM-truenas_core.txt
dmesg_VM-ubuntu_24_04.txt
xl_dmesg_passthrough_disabled-XCPNG_host.txt -
@hvm said in Guest receiving passthrough SATA controllers does not see attached drives:
Hello @Teddy-Astie
Have you had the opportunity to review the log file output?
The past few weeks I have spent a lot of time trying to find a solution. But unfortunately after all the things I have tried, none of them have solved or did changed something to the problem.
We are investigating the possibilities to run our branch locations environments with XCP-NG. This requires a working PoC where passthrough of this SATA controller is necessary.