XCP-ng
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    PCIe Pass-through lanes and lane performance

    Scheduled Pinned Locked Moved Compute
    24 Posts 9 Posters 2.3k Views 7 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M Online
      MajorP93
      last edited by MajorP93

      Hello!
      Just an idea but maybe the PCI link speed you are seeing is related to some sort of energy saving feature?
      Background: I have a Nvidia GPU in my PC and when there is no load the PCIe speed is lowered from PCIe 4.0 to 1.1 due to energy saving feature.

      fbd06c41-50cb-4334-9c98-92a3b6573389-image.jpeg

      So yeah, I think @teddyastie 's idea is a good approach here. Try how your setup behaves during a workload.

      R 1 Reply Last reply Reply Quote 0
      • R Offline
        redakula @MajorP93
        last edited by

        @MajorP93

        I spent quite some time on this when i first got an Arc card and all research i found points to Linux reporting the top of the PCIE chain only. The card has an internal bridge.

        The only tool i found that reports the correct info is GPU-Z which is unfortunately windows only.

        My A310 which also reports PCIE x 1 in linux reports the correct link speed in GPU-Z (It is in a X4 slot so that is why it is not showing the X8 that the card is...)

        02f9582c-2d6c-4647-92ba-6b999c3b436f-image.jpeg

        1 Reply Last reply Reply Quote 0
        • J Offline
          john.c
          last edited by john.c

          Is there anything mentioned in the motherboard or service manuals, about where if one PCIe slot is occupied another is running at reduced speed when occupied?

          Something like this is in the manual for a Gigabyte B450 Aorus Elite (Rev 1.0) motherboard, for example. Can be in yours, somewhere written on one of its pages.

          J 1 Reply Last reply Reply Quote 0
          • J Offline
            JamesG @john.c
            last edited by

            So I ordered some mini DP dummy plugs and took a look at this further today.

            First thought...Just load Windows on the bare metal and see what the card does.

            No problem. Loaded Win10, loaded the latest drivers, PCIe Gen4x8 lanes. Sweet. At least that works.

            Put the XCP-ng drive back in, booted up, passed-through the GPU to a Win11 guest, updated the drivers, PCIe Gen1x1.

            Ultimately I need to install a Linux distro with a really current kernel on the bare metal and see how it sees the GPU. That might be a good project for tomorrow.

            I think this just comes down to a combination of too new, not well supported hardware with some quirks and somewhat dated code and support in XCP-ng (ReBAR support in guests for example).

            @redakula Is that GPU-z screenshot from a Windows guest VM with the A310 passed through?

            Y TeddyAstieT 2 Replies Last reply Reply Quote 0
            • Y Offline
              yannsionneau Vates 🪐 XCP-ng Team @JamesG
              last edited by

              Maybe let's ping the entire @Team-Hypervisor-Kernel here 🙂

              J 1 Reply Last reply Reply Quote 0
              • J Offline
                JamesG @yannsionneau
                last edited by

                @yannsionneau I appreciate that. I will gladly provide access to the hardware if anybody in development wants to dig in and see what's going on.

                1 Reply Last reply Reply Quote 0
                • TeddyAstieT Offline
                  TeddyAstie Vates 🪐 XCP-ng Team Xen Guru @JamesG
                  last edited by TeddyAstie

                  @JamesG said:

                  So I ordered some mini DP dummy plugs and took a look at this further today.

                  First thought...Just load Windows on the bare metal and see what the card does.

                  No problem. Loaded Win10, loaded the latest drivers, PCIe Gen4x8 lanes. Sweet. At least that works.

                  Put the XCP-ng drive back in, booted up, passed-through the GPU to a Win11 guest, updated the drivers, PCIe Gen1x1.

                  As I explained briefly in https://xcp-ng.org/forum/post/106479, this is mostly display as the real hardware may be running at a different speed.

                  Ultimately I need to install a Linux distro with a really current kernel on the bare metal and see how it sees the GPU. That might be a good project for tomorrow.

                  I think this just comes down to a combination of too new, not well supported hardware with some quirks and somewhat dated code and support in XCP-ng (ReBAR support in guests for example).

                  I did quick checks and I believe that ReBAR works; at least with UEFI guests (our OVMF build does support resizable bar), and it actually works as suggest https://xcp-ng.org/forum/post/106489 (VRAM BAR is 16 GB); but I didn't do extensive testing in that area.

                  J 1 Reply Last reply Reply Quote 0
                  • J Offline
                    JamesG @TeddyAstie
                    last edited by

                    Alright....I might be seeing something here...From the CLI of the XCP-ng host itself...

                    Here's the lspci tree view of interest:

                     +-[0000:80]-+-00.0  Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex
                     |           +-00.2  Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU
                     |           +-01.0  Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
                     |           +-01.1-[81-84]----00.0-[82-84]--+-01.0-[83]----00.0  Intel Corporation Device e212
                     |           |                               \-02.0-[84]----00.0  Intel Corporation Device e2f7
                     
                    

                    So PCI slots 81-84 are what I'm interested in...Digging in on 81:

                    lspci -vv -s 0000:81:00.0
                    81:00.0 PCI bridge: Intel Corporation Device e2ff (rev 01) (prog-if 00 [Normal decode])
                            Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
                            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
                            Latency: 0, Cache Line Size: 64 bytes
                            Interrupt: pin A routed to IRQ 56
                            Region 0: Memory at 20003000000 (64-bit, prefetchable) [size=8M]
                            Bus: primary=81, secondary=82, subordinate=84, sec-latency=0
                            Memory behind bridge: f7c00000-f7efffff
                            Prefetchable memory behind bridge: 000001f800000000-0000020002ffffff
                            Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
                            BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
                                    PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
                            Capabilities: [40] Power Management version 3
                                    Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                                    Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
                            Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
                                    Address: 0000000000000000  Data: 0000
                                    Masking: 00000000  Pending: 00000000
                            Capabilities: [70] Express (v2) Upstream Port, MSI 00
                                    DevCap: MaxPayload 256 bytes, PhantFunc 0
                                            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ SlotPowerLimit 75.000W
                                    DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
                                            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop-
                                            MaxPayload 256 bytes, MaxReadReq 512 bytes
                                    DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr+ TransPend-
                                    LnkCap: Port #0, Speed unknown, Width x8, ASPM L1, Exit Latency L0s <4us, L1 <32us
                                            ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
                                    LnkCtl: ASPM Disabled; Disabled- CommClk+
                                            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                                    LnkSta: Speed 16GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                                    DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported
                                    DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                                    LnkCtl2: Target Link Speed: Unknown, EnterCompliance- SpeedDis-
                                             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                                             Compliance De-emphasis: -6dB
                                    LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+, EqualizationPhase1+
                                             EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
                            Capabilities: [100 v2] Advanced Error Reporting
                                    UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                                    UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                                    UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                                    CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                                    CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                                    AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
                            Capabilities: [158 v1] #19
                            Capabilities: [178 v1] #26
                            Capabilities: [1a0 v1] #27
                            Capabilities: [1c8 v1] #2a
                            Capabilities: [1fc v1] Latency Tolerance Reporting
                                    Max snoop latency: 0ns
                                    Max no snoop latency: 0ns
                            Capabilities: [204 v1] L1 PM Substates
                                    L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
                                              PortCommonModeRestoreTime=10us PortTPowerOnTime=14us
                            Capabilities: [220 v1] Vendor Specific Information: ID=0002 Rev=4 Len=100 <?>
                            Capabilities: [320 v1] Vendor Specific Information: ID=0001 Rev=1 Len=038 <?>
                            Capabilities: [358 v1] #25
                            Kernel driver in use: pcieport
                    

                    Specifically there I see the correct width and speed of the PCIe interface for the card.

                    Nothing shows up for 82 (I presume that's a bridge to get to the functions on the card)

                    83 is the actual graphics processor itself:

                    lspci -vv -s 0000:83:00.0
                    83:00.0 VGA compatible controller: Intel Corporation Device e212 (prog-if 00 [VGA controller])
                            Subsystem: Intel Corporation Device 1114
                            Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
                            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
                            Region 0: Memory at 20002000000 (64-bit, prefetchable) [disabled] [size=16M]
                            Region 2: Memory at 1f800000000 (64-bit, prefetchable) [disabled] [size=16G]
                            Expansion ROM at f7c00000 [disabled] [size=2M]
                            Capabilities: [40] Vendor Specific Information: Len=0c <?>
                            Capabilities: [70] Express (v2) Endpoint, MSI 00
                                    DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
                                            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
                                    DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
                                            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
                                            MaxPayload 256 bytes, MaxReadReq 512 bytes
                                    DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                                    LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                                            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                                    LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
                                            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                                    LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
                                    DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR+, OBFF Not Supported
                                    DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                                    LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                                             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                                             Compliance De-emphasis: -6dB
                                    LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                                             EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
                            Capabilities: [ac] MSI: Enable- Count=1/1 Maskable+ 64bit+
                                    Address: 0000000000000000  Data: 0000
                                    Masking: 00000000  Pending: 00000000
                            Capabilities: [d0] Power Management version 3
                                    Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
                                    Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
                            Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
                                    ARICap: MFVC- ACS-, Next Function: 0
                                    ARICtl: MFVC- ACS-, Function Group: 0
                            Capabilities: [110 v1] #00
                            Capabilities: [200 v1] Address Translation Service (ATS)
                                    ATSCap: Invalidate Queue Depth: 00
                                    ATSCtl: Enable-, Smallest Translation Unit: 00
                            Capabilities: [420 v1] #15
                            Capabilities: [220 v1] #24
                            Capabilities: [320 v1] Single Root I/O Virtualization (SR-IOV)
                                    IOVCap: Migration-, Interrupt Message Number: 000
                                    IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
                                    IOVSta: Migration-
                                    Initial VFs: 2, Total VFs: 2, Number of VFs: 0, Function Dependency Link: 00
                                    VF offset: 1, stride: 1, Device ID: e212
                                    Supported Page Size: 00000553, System Page Size: 00000001
                                    Region 0: Memory at 0000020000000000 (64-bit, prefetchable)
                                    Region 2: Memory at 000001fc00000000 (64-bit, prefetchable)
                                    VF Migration: offset: 00000000, BIR: 0
                            Capabilities: [400 v1] Latency Tolerance Reporting
                                    Max snoop latency: 0ns
                                    Max no snoop latency: 0ns
                            Kernel driver in use: pciback
                    

                    And 84 is the audio portion of the GPU...

                    lspci -vv -s 0000:84:00.0
                    84:00.0 Audio device: Intel Corporation Device e2f7
                            Subsystem: Intel Corporation Device 1114
                            Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
                            Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
                            Region 0: Memory at f7e00000 (64-bit, non-prefetchable) [disabled] [size=16K]
                            Capabilities: [50] Power Management version 3
                                    Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
                                    Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
                            Capabilities: [c0] Vendor Specific Information: Len=14 <?>
                            Capabilities: [60] MSI: Enable- Count=1/1 Maskable- 64bit+
                                    Address: 0000000000000000  Data: 0000
                            Capabilities: [80] Express (v2) Endpoint, MSI 00
                                    DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
                                            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
                                    DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
                                            RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
                                            MaxPayload 256 bytes, MaxReadReq 512 bytes
                                    DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
                                    LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
                                            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                                    LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
                                            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                                    LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                                    DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR+, OBFF Not Supported
                                    DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
                                    LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
                                             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                                             Compliance De-emphasis: -6dB
                                    LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                                             EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
                            Capabilities: [100 v1] Advanced Error Reporting
                                    UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                                    UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                                    UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
                                    CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                                    CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                                    AERCap: First Error Pointer: 1f, GenCap- CGenEn- ChkCap- ChkEn-
                            Capabilities: [140 v1] Latency Tolerance Reporting
                                    Max snoop latency: 0ns
                                    Max no snoop latency: 0ns
                    

                    As far as XO (from sources) is concerned, I can turn on pass-through for 83, but it doesn't show me 81, 82, or 84.

                    Should I just manually block off 81-84 and pass all of those devices to the guest VM?

                    J 1 Reply Last reply Reply Quote 0
                    • J Offline
                      JamesG @JamesG
                      last edited by

                      Argh...Correction... 82 shows up as 82:01 and 82:02...

                      lspci -vv -s 0000:82:01.0
                      82:01.0 PCI bridge: Intel Corporation Device e2f0 (prog-if 00 [Normal decode])
                              Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
                              Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
                              Latency: 0, Cache Line Size: 64 bytes
                              Bus: primary=82, secondary=83, subordinate=83, sec-latency=0
                              Memory behind bridge: f7c00000-f7dfffff
                              Prefetchable memory behind bridge: 000001f800000000-0000020002ffffff
                              Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
                              BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
                                      PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
                              Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00
                                      DevCap: MaxPayload 256 bytes, PhantFunc 0
                                              ExtTag+ RBE+
                                      DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
                                              RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop-
                                              MaxPayload 256 bytes, MaxReadReq 512 bytes
                                      DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                                      LnkCap: Port #8, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L0s <64ns, L1 <1us
                                              ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                                      LnkCtl: ASPM Disabled; Disabled- CommClk-
                                              ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                                      LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                                      DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd+
                                      DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd+
                                      LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
                                               Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                                               Compliance De-emphasis: -6dB
                                      LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                                               EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
                              Capabilities: [a0] Power Management version 3
                                      Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                                      Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
                              Capabilities: [94] Subsystem: Intel Corporation Device 0000
                              Capabilities: [100 v2] Advanced Error Reporting
                                      UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                                      UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                                      UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
                                      CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                                      CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                                      AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
                              Capabilities: [220 v1] Access Control Services
                                      ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+
                                      ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
                              Kernel driver in use: pcieport
                      

                      And 82:02...

                      lspci -vv -s 0000:82:02.0
                      82:02.0 PCI bridge: Intel Corporation Device e2f1 (prog-if 00 [Normal decode])
                              Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
                              Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
                              Latency: 0, Cache Line Size: 64 bytes
                              Bus: primary=82, secondary=84, subordinate=84, sec-latency=0
                              Memory behind bridge: f7e00000-f7efffff
                              Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
                              BridgeCtl: Parity- SERR+ NoISA- VGA- MAbort- >Reset- FastB2B-
                                      PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
                              Capabilities: [40] Express (v2) Downstream Port (Slot-), MSI 00
                                      DevCap: MaxPayload 256 bytes, PhantFunc 0
                                              ExtTag+ RBE+
                                      DevCtl: Report errors: Correctable- Non-Fatal- Fatal+ Unsupported-
                                              RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop-
                                              MaxPayload 256 bytes, MaxReadReq 512 bytes
                                      DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- TransPend-
                                      LnkCap: Port #8, Speed 2.5GT/s, Width x1, ASPM L1, Exit Latency L0s <64ns, L1 <1us
                                              ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                                      LnkCtl: ASPM Disabled; Disabled- CommClk+
                                              ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                                      LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                                      DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd+
                                      DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
                                      LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
                                               Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                                               Compliance De-emphasis: -6dB
                                      LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
                                               EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
                              Capabilities: [a0] Power Management version 3
                                      Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
                                      Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
                              Capabilities: [94] Subsystem: Intel Corporation Device 0000
                              Capabilities: [100 v2] Advanced Error Reporting
                                      UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                                      UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                                      UESvrt: DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol-
                                      CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                                      CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
                                      AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-
                              Capabilities: [220 v1] Access Control Services
                                      ACSCap: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans+
                                      ACSCtl: SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
                              Kernel driver in use: pcieport
                      
                      poddingueP 1 Reply Last reply Reply Quote 0
                      • poddingueP Offline
                        poddingue Vates 🪐 @JamesG
                        last edited by

                        Before you go further down the bridge tree, redakula called this one early in the thread and it's worth rereading.

                        Intel Arc cards put an internal PCIe bridge between the slot and the GPU, and lspci on Linux only reports the top of that chain. His own A310 shows PCIe x8 4.0 at x4 4.0 in GPU-Z under Windows while Linux insists it is x1. That is the same bridge topology you're now staring at on 81, 82 and 83.

                        TeddyAstie said much the same when he called it mostly a display issue, and he later checked that resizable BAR does work on UEFI guests since the OVMF build supports it. So the Gen1x1 reading may just be lspci describing the bridge rather than the card. I could easily be wrong, and the people already in this thread know this hardware far better than I do. 🤷

                        1 Reply Last reply Reply Quote 0

                        Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                        Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                        With your input, this post could be even better 💗

                        Register Login
                        • First post
                          Last post