2
\$\begingroup\$

My understanding of PCIe bridges is as follows:

A PCIe bridge appears as 1 device on the upstream bus, creates a new bus downstream of it, and whatever connects to the downstream side will appear as the first (and only) device on that bus.

A PCIe switch could be attached to the downstream side of a bridge, in which case it would appear as a device on the PCIe bridge's bus, and it would also create its own internal bus. All of its downstream bridges would then appear as devices on the switch's internal bus, while each creating their own buses for whatever may attach to them.

However, in analyzing the output of lspci and lspci -t from my system, I'm encounter what appears to be a PCIe bridge that makes available 2 functions on the upstream bus it is attached to. Downstream, 2 switches connect to each function (1 each) and make their own buses which each of their own downstream bridges appear as devices on.

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne Root Complex
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne IOMMU
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe GPP Bridge
00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge
...
01:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch (rev c0)
02:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] (rev c0)
...
04:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream
05:01.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
05:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
05:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
05:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
05:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
05:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge
...

lspci -t

-[0000:00]-+-00.0
           +-00.2
           +-01.0
           +-01.1-[01-03]----00.0-[02-03]----00.0-[03]--+-00.0
           |                                            \-00.1
           +-01.2-[04-0b]----00.0-[05-0b]--+-01.0-[06]----00.0
...

Here, Device 01:00.0 is an upstream port for the switch, meaning its residing on the bus made available by the bridge 01.1 named "Renoir PCIe GPP Bridge".

Its own internal bus is bus 02. As seen by the 02:00.0, "Downstream Port of PCI Express Switch", which makes its own bus 03 for the device connected to it.

However, Device 04:00.0 is another upstream switch that is connected to the same device on Bus 0, but with a different function.

I'm not sure how to interpret these results because I've not seen any indication that bridges offer this type of branching functionality. So, is this simply a case of not understanding the technical definition of a bridge and knowing its full capabilities? Or is there something else going on here that I'm not understanding?

One possible explanation based on definitions from the PCIe spec is that a single Device is not necessarily one physical device in PCIe (whereas I believe this is the case in PCI) and these are 2 separate "physical bridges" being mapped as a single device.

Bridge: One of several defined System Elements. A Function that virtually or actually connects a PCI/PCI-X segment or PCI Express Port with an internal component interconnect or with another PCI/PCI-X bus segment or PCI Express Port. A virtual Bridge in a Root Complex or Switch must use the software configuration interface described in this specification.

Device: A collection of one or more Functions within a single Hierarchy identified by common Bus Number and Device Number. An SR-IOV Device may have additional Functions accessed via additional Bus Numbers configured through10 one or more SR-IOV Capability structures.

But I've not found other documentation to support this.

\$\endgroup\$

2 Answers 2

Reset to default
1
\$\begingroup\$

You are confusing functions and downstream connections. Your interpretation about functions and lspci are both incorrect.

Functions:

It should be made clear that functions is not something that makes a device (in this case bridge) connect with other device. Function is just about the device itself and what it can do. There is a configuration memory related to every function. And it is must for every device to have at least on function.

\$\endgroup\$
0
\$\begingroup\$

No! A PCIe Bridge cannot connect to multiple downstream devices using different functions. Infact, the connection between a PCIe bridge and downstream devices is established through PCIe slots or ports, not through the functions of the bridge itself. Your interpretation is incorrect.

Details: A PCIe bridge can connect to multiple downstream devices, and each connected device may have multiple functions. In the PCIe architecture, the concept of functions refers to different capabilities or logical divisions within a single device and the number of functions within a PCIe bridge does not directly correspond to the number of devices it can connect to.

From the example that you have provided:

00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe Dummy Host Bridge
00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir PCIe GPP Bridge
00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge

This show that there is a bridge 00:01.x connected to the root complex that has 3 functions (capabilities). This is just one bridge and it is not describing any device connected to it. It is just describing the bridge itself that has 3 capabilities.

03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] (rev c0)

And this indicates that this is an endpoint device connected at 03:00.x that has only one capability.

Hope that helps

\$\endgroup\$

Not the answer you're looking for? Browse other questions tagged or ask your own question.