XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Guest receiving passthrough SATA controllers does not see attached drives

    Scheduled Pinned Locked Moved Compute
    4 Posts 2 Posters 410 Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H Offline
      hvm
      last edited by olivierlambert

      Re: Guest receiving passthrough SATA controllers does not see attached drives

      It seems i'm having the same issue, using an AMD Epyc 7313P with Asrock Rack ROMED6U-2L2T motherboard and also having problem with passthrough the Advanced Micro Devices, Inc. [AMD] FCH SATA Controllers.

      *01:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
        Subsystem: ZyDAS Technology Corp. ASM1166 Serial ATA Controller
        Flags: bus master, fast devsel, latency 0, IRQ 132, IOMMU group 74
        Memory at f8982000 (32-bit, non-prefetchable) [size=8K]
        Memory at f8980000 (32-bit, non-prefetchable) [size=8K]
        Expansion ROM at f8900000 [disabled] [size=512K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [80] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [130] Secondary PCI Express
        Kernel driver in use: ahci
        Kernel modules: ahci
45:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
        Flags: bus master, fast devsel, latency 0, IRQ 74, IOMMU group 59
        Memory at b2400000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+
        Capabilities: [d0] SATA HBA v1.0
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] Transaction Processing Hints
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: ahci
        Kernel modules: ahci

46:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode]
        Flags: bus master, fast devsel, latency 0, IRQ 99, IOMMU group 60
        Memory at b2300000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+
        Capabilities: [d0] SATA HBA v1.0
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] Transaction Processing Hints
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: ahci
        Kernel modules: ahci*

*45:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
        Subsystem: Advanced Micro Devices, Inc. [AMD] Device 7901
        Flags: bus master, fast devsel, latency 0, IRQ 105
        Memory at b2400000 (32-bit, non-prefetchable)
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+
        Capabilities: [d0] SATA HBA v1.0
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] Transaction Processing Hints
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>

46:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 51) (prog-if 01 [AHCI 1.0])
        Subsystem: Advanced Micro Devices, Inc. [AMD] Device 7901
        Flags: bus master, fast devsel, latency 0, IRQ 109
        Memory at b2300000 (32-bit, non-prefetchable)
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=16/16 Maskable- 64bit+
        Capabilities: [d0] SATA HBA v1.0
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] Transaction Processing Hints
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>*

      When I passthrough the PCIe ASM1166 card, can see and use the drives (that are connected to this card) within the VM.
      But when I passthrough the Advanced Micro Devices, Inc. [AMD] FCH SATA Controllers [AHCI mode], don't see any connected drives to these controllers.
      In the guest os (tested with Ubuntu and Truenas Core & Scale) i'm getting drive error.

      For example, this are a few errors i get in Truenas Core:

      
*root@truenas[~]# camcontrol reportluns 0:0:0
(pass0:ahcich0:0:0:0): REPORT LUNS. CDB: a0 00 00 00 00 00 00 00 00 10 00 00
(pass0:ahcich0:0:0:0): CAM status: CCB request was invalid

root@truenas[~]# camcontrol epc ada0 -c list
(pass0:ahcich0:0:0:0): READ_LOG_DMA_EXT. ACB: 47 00 08 00 00 40 00 00 00 00 02 00
(pass0:ahcich0:0:0:0): CAM status: ATA Status Error
(pass0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT )
(pass0:ahcich0:0:0:0): RES: 51 04 08 00 00 40 00 00 00 02 00

root@truenas[~]# pciconf -lv
ahci2@pci0:1:0:0:       class=0x010601 rev=0x02 hdr=0x00 vendor=0x1b21 device=0x1166 subvendor=0x2116 subdevice=0x2116
    vendor     = 'ASMedia Technology Inc.'
    class      = mass storage
    subclass   = SATA
ahci0@pci0:69:0:0:      class=0x010601 rev=0x51 hdr=0x00 vendor=0x1022 device=0x7904 subvendor=0x1022 subdevice=0x7901
    vendor     = 'Advanced Micro Devices, Inc. [AMD]'
    device     = 'FCH SATA Controller [AHCI mode]'
    class      = mass storage
    subclass   = SATA
ahci1@pci0:70:0:0:      class=0x010601 rev=0x51 hdr=0x00 vendor=0x1022 device=0x7904 subvendor=0x1022 subdevice=0x7901
    vendor     = 'Advanced Micro Devices, Inc. [AMD]'
    device     = 'FCH SATA Controller [AHCI mode]'
    class      = mass storage
    subclass   = SATA*

      When passthrough is disabled, i can see and use the connected disks just fine within XCP-NG and then no errors are reported.

      Any thoughts?

      1 Reply Last reply Reply Quote 0
      • TeddyAstieT Online
        TeddyAstie Vates 🪐 XCP-ng Team Xen Guru
        last edited by

        Hello @hvm,

        Can you give the output of xl dmesg in XCP-ng and of dmesg in the guest that has the issues ?
        I have the impression that something is going wrong with reserved regions related to the SATA controller.

        H 1 Reply Last reply Reply Quote 1
        • H Offline
          hvm @TeddyAstie
          last edited by hvm

          Hi @Teddy-Astie

          First of all, thanks for your thoughts!
          I'm pretty stuck with these issues. So hopefully other insights will help 🙂
          Below are the requested log files.

          xl_dmesg_passthrough_enabled-XCPNG_host.txt
          pci_devices-XCPNG_host.txt
          dmesg_VM-truenas_core.txt
          dmesg_VM-ubuntu_24_04.txt
          xl_dmesg_passthrough_disabled-XCPNG_host.txt

          H 1 Reply Last reply Reply Quote 0
          • H Offline
            hvm @hvm
            last edited by

            @hvm said in Guest receiving passthrough SATA controllers does not see attached drives:

            @Teddy-Astie

            Hello @Teddy-Astie

            Have you had the opportunity to review the log file output?

            The past few weeks I have spent a lot of time trying to find a solution. But unfortunately after all the things I have tried, none of them have solved or did changed something to the problem.

            We are investigating the possibilities to run our branch locations environments with XCP-NG. This requires a working PoC where passthrough of this SATA controller is necessary.

            1 Reply Last reply Reply Quote 0
            • First post
              Last post