Recent Altera® FPGAs support PCI Express (PCIe) as standard, and many people wonder where to start when they want to implement PCI Express in an Altera® FPGA. Many people have wondered where to start when trying to implement PCI Express in an Altera® FPGA. For example,
What FPGA devices support what?
What transfer rates are supported by each FPGA device?
What is the FPGA design methodology? What reference designs are available?
What FPGA devices support what transfer rate?
Tools may include Altera® Quartus® Prime development software and, in some cases, the Platform Designer system integration tool bundled with Quartus® Prime.
For those new to PCI Express with Altera® FPGAs, here is a step-by-step process from selection to an overview of the design methodology.
Step 1: Selecting the device to use
To get started, visit the Altera® FPGA site at PCI-Express Protocol page on the Altera® FPGA site. Here you will find an overview of the PCI Express protocol and an overview of Altera® FPGA's PCI Express solution, with sections on features, device support, getting started guide, and technical support.
PCI Express Protocol
https://www.intel.com/content/www/us/en/programmable/products/intellectual-property/ip/interface-protocols/m-pci- express-protocol.html
The Device Support section allows you to see which devices can be used to achieve the configuration you want to achieve. For example, a PCI Express Gen1 x4 configuration can be achieved with all devices listed. If you want to realize a Gen3 x8 configuration, the number of devices that can be used is limited.
Also, in the "Device Configuration and Feature Support" section, there is mention of endpoints and root ports, but Altera® FPGA PCI Express IP can realize both endpoints and root ports (ports when implementing a root complex). In the case of a root complex, it is possible to implement a root port. A root complex can be implemented by setting the IP core to the root port and adding control by the host controller.
PCI Express Configuration Diagram
Step 2: Determine the configuration in the FPGA by the data transfer format
The documentation list in the Getting Started Guide section shows that there is one for Avalon-Streaming (Avalon-ST), one for Avalon-Memory Mapped (Avalon-MM), and one for Avalon-MM with DMA, but which one to choose depends on your usage conditions. The choice depends on the conditions of use. The differences are briefly explained below.
If you are interested in Avalon-ST or Avalon-MM, please refer to this page.
FAQ] What is the difference between the Avalon-MM interface and the Avalon-ST interface?
Avalon-ST Configuration
The interface on the FPGA fabric side of PCI Express is Avalon-ST, which transfers data in a stream format with Start-of-Packet (SOP) and End-of-Packet (EOP) indicating the start and end of data. The data is transmitted in a stream. The data is a PCI Express TLP (Transaction Layer Packet) itself, and the details of the TLP must be understood and implemented according to the PCI Express specification. There are no other specification restrictions; implementation within Platform Designer or without Platform Designer is possible.
Avalon-ST Configuration
Avalon-MM configuration
The interface on the FPGA fabric side of PCI Express is Avalon-MM, which transfers data in memory-mapped format and requires addressing when sending and receiving data. The use of Platform Designer is mandatory for implementation, and address conversion processing is required between the Platform Designer internal address space and the address space of the host PC. However, it is relatively easy to implement without understanding the details of TLP. Limitations of the specification include: less tag support than the Avalon-ST configuration, single function support only, no ECRC forwarding support, and no Expansion ROM support.
Avalon-MM Configuration
Avalon-MM with DMA Configuration
The basic configuration is the same as the Avalon-MM configuration, but a DMA engine is implemented in the core for more efficient transfers. The specification limitations are the same as those of the Avalon-MM configuration, including the inability to implement a root port and the limited support for TLP. For details, please refer to the comparison table in each document.
Avalon-MM with DMA Configuration
Based on the above, decide which configuration to select depending on the case you wish to achieve.
Case 1
In cases where the root port/endpoints are to be realized in a single function, the Avalon-MM configuration is generally used. Access (mSGDMA), which comes standard with Platform Designer, can also be used for fast DMA transfers. Although there are some limitations, such as limited tag support, single-function support only, no ECRC forwarding support, and no Expansion ROM support, this configuration is recommended as long as it does not conflict with those points.
Case 2
In cases where the root port/endpoint is multi-functional and Platform Designer is not desired, implementation in Avalon-ST configuration is mandatory. The user logic must be able to understand the details of TLP and implement Avalon-ST packets.
Case 3
For example, if you want to achieve the highest possible transfer efficiency in a Gen3 x8 configuration, choose the Avalon-MM with DMA configuration; since the DMA engine is in the Avalon-MM bridge, higher transfer efficiency can be achieved.
Step 3: Try with a Reference Design
Below is a collection of reference designs for the Avalon-MM configuration.
Intel FPGA Wiki: PCI Express in Qsys Example Designs
The following is a design for Avalon-MM with DMA.
Cyclone_V_Gen2x4_AVMM_DMA
Cyclone_V_Gen2x4_AVMM_DMA_external_DDR3
Arria_V_Gen2x4_AVMM_DMA
Arria_V_Gen2x4_AVMM_DMA_external_DDR3
Arria_10_Gen3x8_AVMM_DMA
Arria_10_Gen3x8_AVMM_DMA_external_DDR4
Stratix_10_Gen3x8_AVMM_DMA
Reference Information
Avalon-ST Configuration Implementation Documentation
You can use any of the reference designs that can be generated by Quartus® Prime. From the Example Designs tab, a Quartus® Prime project is generated. This design will be an Avalon-ST configuration.
PCI Express with Altera® FPGAs (Avalon-ST Edition)
Implementation Documentation for Avalon-MM Configuration
PCI Express in Altera® FPGAs (Avalon-MM Edition)
Implementing DMA Transfers Using PCI Express Hard IP
Recommended articles and documents
PCI Express in Altera® FPGAs (Avalon-ST Edition)
PCI Express in Altera® FPGAs (Avalon-MM Edition)
PCI Express Design & Debug Guidelines for FPGAs