Adding More Ethernet Ports to Raspberry Pi CM5 using PCIe Bus

I was asked often to design custom IO boards for Raspberry Pi Compute Module with multiple Ethernet ports. Before, I used USB Ethernet controllers or Ethernet switch ICs on CM5’s native Ethernet port. These approaches were easy to implement, but were only suitable for low-speed Ethernet applications only. The most recent one was to add 2 x 1Gb Ethernet ports that are capable of trans/receiving Jumbo frames. I added two Ethernet controller ICs via PCIe bus, thus, the board is having total three 1Gb Ethernet ports.

Key Features

• Raspberry Pi CM5 placeholder
• 1 x HDMI 2.0 output
• 1 x USB 3.0 Type-A
• 1 x USB 2.0 Type-A
• 3 x 1Gb Ethernet
  • 1 x 1Gb Ethernet from CM5
  • 4-way PCIe Gen2 switch, ASmedia ASM1184e
    • 1 x 1Gb Ethernet from Realtek RTL8111 controller
    • 1 x 1Gb Ethernet from Realtek RTL8111 controller
    • 2 x PCIe ports left unused
• 40p GPIO header, compatible with Raspberry Pi’s standard
• 1 x USB 2.0 Type-C port for USB PD power supply and OS image flashing onto eMMC of CM5

PCB Design and PCBA Prototyping

Just like all my other recent designs, I designed this board using EasyEDA Pro.

I have designed PCIe Gen2 switch for Raspberry Pi CM4/CM5 carrier boards before several times. So it was not much difficult for this board. But designing the 1Gb Ethernet via PCIe was my very first trying. After some research, I decided to use Realtek RTL8111, which seemed like a perfect fit for the requirements – PCe Gen2 and 1Gb Ethernet. And it was well available on LCSC at the time of designing.

There was less public references of sample schematics for this specific RTL8111H. So I had to design the circuit mostly referencing the datasheet only. However, it was a lot similar to the other Realtek’s Ethernet ICs like RTL8152B, which I’ve used many times for adding low-speed multiple Ethernet ports.

The RTL8111H had two variants – RTL8111H and RTL8111HS. They where having different method of 1.0V power supply voltage generating. It was somewhat similar to the difference between RP2040 and RP2350, having LDO and switch mode regulator. Although I used RTL8111H, which is having LDO, I added a small note to not confuse myself in the future.

The PCIe Gen 2.0 switch IC, ASmedia ASM1184e was unavailable on LCSC, Digi-Key and Mouser at this time of designing. But it was well available in Chinese local market. In such cases, using JLCPCB’s online website becomes complex process, for purchasing chips from local vendors, asking them to JLCPCB fab

I recommended PCB-Hero, which is a custom PCBA company, which I used to order troublesome PCB designs often. As usual, they managed to source ASM1184e PCIe Gen2 Switch chip and assembled the whole board quickly. Although the pricing of PCBA on PCB-Hero was a little more expensive than JLCPCB, I always think it’s well worth of the work.

Testing

I didn’t get the physical boards on my hand. Instead I asked PCB-Hero to ship the prototyped 2 boards directly to my client. He tested it and we could see all added two Ethernet ports are working at full 1Gb speed.

For enabling Jumbo frames on these RTO8111 Ethernet ports, we would need to use custom drivers. We could see some forum posts related to the topic. Although the Jumbo frames are not tested yet, I believe that we can get it working soon, as it is completely software related work.