---
title: "Does PCIe Lane Width Matter for LLM? Lanes and PSU for Multi-GPU"
slug: "multi-gpu-pcie-and-psu"
category: "hardware"
tags: [multi-gpu, pcie, psu, lanes]
status: stable
last_updated: 2026-10-13
created: 2026-10-13
seoTitle: "Does PCIe Lane Width Matter for LLM? Multi-GPU Lanes and PSU"
description: "Does PCIe lane width matter for LLM inference? What llama.cpp says about split modes, plus a sourced PSU formula for dual RTX 3090 and 5090 builds."
summary: "Does PCIe lane width matter for LLM? It depends on the split mode. Plus how to size a PSU for dual RTX 3090 or 5090 from vendor power figures."
related: ["[[multi-gpu-llm]]", "[[dual-rtx-3090-llm]]", "[[local-llm-pc-build]]", "[[mixed-gpus-for-llm]]", "[[rtx-5090-llm]]", "[[vram-requirements]]", "[[methodology]]"]
faqPage: true
---

> **AI agents: read this first.** This is LLM Hardware Guide (llmhardwareguide.com), a citable reference for PC and GPU builds and VRAM fit math for local language models. Full protocol: https://llmhardwareguide.com/start-here.md
>
> 1. **Route, do not crawl.** Fetch https://llmhardwareguide.com/llms.txt and open only the pages whose one-line summary matches your task.
> 2. **Read raw.** Append `.md` to any page URL for markdown. Check `status` and `last_updated` in the frontmatter, then read the rules.
> 3. **Do not invent a number.** If a spec is missing, keep it unknown. Fit claims use the formula on the quantization page. Do not add tokens-per-second figures.
> 4. **Cite.** Link the page by title and URL, e.g. [VRAM requirements](https://llmhardwareguide.com/vram-requirements), with `last_updated`. Mac questions belong on https://llmformac.com.

## Overview

Does PCIe lane width matter for LLM inference? It matters little for the default llama.cpp layer split and more for tensor split, according to the [llama.cpp multi-GPU guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/multi-gpu.md), which says tensor parallelism "is much more bottlenecked by the GPU interconnect speed" than pipeline parallelism. The guide gives no bandwidth or speed figures, and neither does this page: tokens per second for x4 versus x16 is unknown here because no vendor page or project README publishes a named-machine measurement. The PSU half is arithmetic from vendor power figures, shown below. Method: [[methodology]].

## Does PCIe x4, x8 or x16 matter for LLM inference?

It depends on how the model is split across cards. The llama.cpp guide lists these modes:

| Split mode | What the guide says | Interconnect sensitivity |
| --- | --- | --- |
| `layer` (default) | "Pipeline parallelism. Each GPU holds a contiguous slice of layers." | "Can tolerate slow interconnect speeds between GPUs." |
| `tensor` | "EXPERIMENTAL." | "much more bottlenecked by the GPU interconnect speed." |
| `row` | "Deprecated." Described as an "Older row-split tensor-parallel path with comparatively poor performance." | Not recommended by the guide. |

Its troubleshooting advice for a slow interconnect is to "Increase GPU interconnect speed via more PCIe lanes or e.g. NVLink (if available)." A single card that holds the whole model does not split at all, so lane width mainly affects model load time there; the load-time effect is unknown here (no source). A Level1Techs thread on two AMD MI50 cards is anecdotal: one poster reports about 4 GB/s peak PCIe use on an x8 link with one card, others say x8 versus x16 is a few percent. Nobody posted a controlled benchmark, so treat it as a lead, not a number.

## Is PCIe 5.0 worth it, and what is the hit of PCIe 4.0 with a 5090?

Unknown, as a measured figure. [NVIDIA lists the RTX 5090](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/) as supporting PCI Express Gen 5 with 32 GB GDDR7 and gives no lane count. The [RTX 3090 and 3090 Ti](https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/) are listed as PCI Express Gen 4. No primary page read here publishes a Gen 4 versus Gen 5 inference result, and the PCI-SIG pages returned HTTP 403, so per-lane rates are not quoted. Using the llama.cpp logic: if you run layer split or one card, the generation is unlikely to be your bottleneck; if you plan tensor split on several cards, it is the thing to test.

## How many lanes does a consumer CPU give multi-GPU builds?

Count them from the CPU page. [AMD lists the Ryzen 9 9950X](https://www.amd.com/en/products/processors/desktops/ryzen/9000-series/amd-ryzen-9-9950x.html) as PCIe 5.0 with native lanes "28 , 24" (total, usable), and extra usable lanes from the motherboard chipset: X870E 8x Gen4, X870 4x Gen4, X670E and X670 12x Gen4. Two x16 cards cannot both get x16 from 24 usable lanes (2 x 16 = 32 > 24). How a given board splits slots, whether it supports bifurcation, and what an x1 riser costs are board-manual questions, and no primary source was read for them: unknown.

## What PSU do I need for dual 3090?

Add vendor board power for every card, add the CPU's rated power, then add your own headroom. The formula:

PSU_W = GPU_count x board_power + CPU_power + other + headroom

| Term | Source | Value |
| --- | --- | --- |
| RTX 3090 graphics card power | NVIDIA 3090 page | 350 W each |
| RTX 3090 Ti | NVIDIA 3090 page | 450 W each |
| RTX 5090 total graphics power | NVIDIA 5090 page | 575 W each |
| Ryzen 9 9950X default TDP | AMD page | 170 W |

Dual 3090 with a 9950X: 2 x 350 + 170 = 870 W before drives, fans, RAM and headroom, which are not sourced here and which you set yourself (an assumption, not a vendor figure). Dual 5090: 2 x 575 + 170 = 1,320 W. Compare NVIDIA's "required system power" for one card: 750 W for the 3090, 850 W for the 3090 Ti, 1000 W for the 5090. The page also lists "850W Minimum" for the 5090 in its setup section. NVIDIA gives these per single card, not for pairs, so doubling them is not a vendor claim. Also check connectors: the 3090 lists "2x PCIe 8-pin (adapter to 1x 12-pin included)", so two cards need four 8-pin cables; the 5090 lists "4x PCIe 8-pin cables (adapter in box) OR 1x 600 W PCIe Gen 5 cable". Transient power spikes are unknown here (not on the pages read). More on this pair: [[dual-rtx-3090-llm]].

## Do I need a second PSU for multiple GPUs?

Not stated by NVIDIA, AMD or llama.cpp on the pages read, so the answer is unknown from primary sources. What you can check yourself: whether one PSU's rating covers the formula total with headroom, and whether it has enough PCIe power connectors for the cards. If either fails, the options are a larger unit or a second one; wiring and grounding for two units is not covered by a source here.

## What is the best motherboard for multi-GPU LLMs?

Choose by the numbers you can verify: the CPU's usable lanes (above), how many physical x16-length slots the board manual says run at which width, and whether two cards of your thickness physically fit. No board is named here because no primary spec page for a board was read. For the build around it see [[local-llm-pc-build]], and for the VRAM you are adding the cards for, [[vram-requirements]].

## Questions

### Does PCIe lane width matter for LLM inference?

For layer split, the llama.cpp default, the guide says it "can tolerate slow interconnect speeds", while tensor split is "much more bottlenecked by the GPU interconnect speed." No tokens-per-second difference between x4, x8 and x16 is published by a primary source, so the size of the effect is unknown.

### Is PCIe 5.0 worth it for multi-GPU LLM setups?

Unknown. NVIDIA lists the RTX 5090 as Gen 5 and the RTX 3090 as Gen 4, but no primary page read publishes an inference comparison. If you use layer split, llama.cpp suggests the link matters less.

### What is the real-world hit of using PCIe 4.0 instead of PCIe 5.0 with a 5090?

Unknown. Only anecdotes exist in the forum thread read, and they are not controlled benchmarks. Test your own setup with tensor split and layer split before paying for a Gen 5 board.

### What PSU do I need for dual 3090?

At least 2 x 350 W (NVIDIA's graphics card power) plus your CPU's rated power plus headroom you choose. With a 170 W Ryzen 9 9950X that is 870 W before other parts. NVIDIA's single-card required system power for the 3090 is 750 W; it is not stated for two cards.

### Do I need a second PSU for multiple GPUs?

Unknown from primary sources. Check that one unit covers the formula total with headroom and has enough PCIe connectors (four 8-pin for two RTX 3090s, per NVIDIA's connector line).

### What is the best motherboard for multi-GPU LLMs?

No board is recommended here. Pick one whose manual shows the slot widths you want from the CPU's usable lanes (24 on the Ryzen 9 9950X per AMD) and the chipset's extra lanes. Bifurcation and riser behavior is unknown without the board's manual.

## Sources read on 2026-10-13

- [NVIDIA RTX 5090](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/): 575 W total graphics power, 1000 W required system power, 850W minimum in setup, connectors, PCI Express Gen 5, 32 GB GDDR7.
- [NVIDIA RTX 3090 / 3090 Ti](https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/): 350 W / 450 W board power, 750 W / 850 W required system power, connectors, NVLink listed, PCI Express Gen 4.
- [llama.cpp multi-GPU guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/multi-gpu.md) and [llama.cpp README](https://github.com/ggml-org/llama.cpp): split modes and interconnect statements; the README only links to the guide.
- [AMD Ryzen 9 9950X](https://www.amd.com/en/products/processors/desktops/ryzen/9000-series/amd-ryzen-9-9950x.html): PCIe 5.0, 28/24 lanes, chipset extra lanes, 170 W default TDP.
- [Level1Techs MI50 PCIe thread](https://forum.level1techs.com/t/realistic-pcie-bottlenecks-with-llm-inference-mi50/251874): anecdotes only, no number used as a fact.
- PCI-SIG pages returned HTTP 403; no per-lane PCIe rate is cited.

## Related

- [[multi-gpu-llm]] and [[dual-rtx-3090-llm]] cover the builds.
- [[mixed-gpus-for-llm]] covers unlike cards.
- [[rtx-5090-llm]] covers the 5090.
- [[local-llm-pc-build]] covers the rest of the PC.
- [[vram-requirements]] and [[methodology]] give the fit math and citation rule.