Dual RTX 3090 for Local LLMs: 48GB Build, NVLink and 70B

Overview

A dual 3090 LLM build gives you two 24 GB cards, 48 GB of VRAM in total, and that is enough for a 70B model at Q4_K_M only barely and only at short context: the 43.1 GB file plus about 1.3 GB of KV cache at 4,096 tokens leaves a thin margin that the assumed 1 to 2 GB overhead per card can use up, and 32,768 tokens does not fit. The 48 GB is two separate pools that the runtime splits the model across, not one unified memory. The fit math follows methodology. Speed in tokens per second is unknown on this page because no vendor or project page publishes a dual 3090 figure.

What does a dual RTX 3090 build give you?

Per NVIDIA’s RTX 3090 page, each card has 24 GB of GDDR6X on a 384-bit interface, so two cards are 2 x 24 = 48 GB. NVIDIA’s page does not state memory bandwidth, so it is unknown here.

The 48 GB is pooled by splitting the model, and llama.cpp names the modes. Per the llama.cpp CLI README, --split-mode layer (the default) will “split layers and KV across GPUs (pipelined)”, row will “split weight across GPUs by rows (parallelized)” (the llama.cpp multi-GPU guide marks row as deprecated), and tensor will “split weights and KV across GPUs (parallelized, EXPERIMENTAL)“. Layer split puts different layers on different cards. The README describes the split but does not say how a single buffer is placed, so treat each card as its own 24 GB pool.

Can two RTX 3090s run a 70B model?

Borderline at Q4_K_M with short context, no at 32k. Weights: the llama.cpp quantize README lists Llama 3.1 70B at 280.9 GB original and 43.1 GB at Q4_K_M. KV cache uses the formula from methodology: 2 x layers x kv_heads x head_dim x context x bytes_per_value. Llama 3.1 70B’s config.json is gated (HTTP 401 when fetched), so this page uses the Qwen2.5-72B model card and config.json as the 70B-class stand-in: 80 layers, 8 KV heads, hidden size 8192 over 64 heads (head dimension 128), bfloat16 (2 bytes). That is 2 x 80 x 8 x 128 x 2 = 327,680 bytes per token.

Overhead of 1 to 2 GB per card is the site’s assumption, not a measurement; the same per-card convention is used on vram-for-70b-model and multi-gpu-llm. Card sizes are NVIDIA’s “24 GB” and the README file size is “43.1 GB”; this page treats both as decimal GB and does not convert units, so leave extra margin. Each card carries its own runtime buffers, which is why the table counts the overhead twice. Treat 4,096 as borderline. The longest context the arithmetic leaves room for is (48 - 43.1 - overhead) / 327,680 bytes per token: about 11,901 tokens at 1 GB overhead, 8,850 at 2 GB, and 2,746 at 4 GB. Q8_0 is not on the table: the README’s Memory/Disk table gives no Q8_0 size for 70B, so it is unknown here. A smaller quant or a 32B model is the route for long context; see vram-for-70b-model and vram-for-32b-model.

Do I need NVLink for two 3090s?

No, you do not need it to run a model, and NVIDIA’s own pages do not tell you it will make LLM inference faster. The RTX 3090 page lists “NVIDIA NVLink (SLI-Ready)” and describes a separate NVLink bridge, “a new, smaller connector to link two NVLink SLI-ready 30 Series graphics cards”. The RTX 5090 page lists NVLink as “No”, so NVLink is a reason some people pick 3090s over 5090s.

For the speed effect, the only primary source read is NVIDIA’s NVLink and NVSwitch blog, which tests Hopper H200 servers, not 3090s. It reports a 1.0x gain at batch 1 (26 versus 25 tok/s/GPU) and 1.5x at batch 32 (168 versus 112), for Llama 3.1 70B at TP=2, and says the benefit grows with batch size. Those numbers are for NVSwitch versus point-to-point on H200 and cannot be carried to a 3090 bridge. The bridge benefit for a single user on two 3090s is unknown. A Hacker News thread, 25L Portable NV-linked Dual 3090 LLM Rig, shows people build it; no figure from it is used here.

Dual 3090 or a single RTX 5090?

For a 70B Q4_K_M file, two 3090s fit and one 5090 does not: the 5090’s 32 GB GDDR7 on a 512-bit interface (per NVIDIA’s page) is less than the 43.1 GB file before any KV cache.

Sources: NVIDIA’s RTX 3090 and RTX 5090 pages. Neither page states bandwidth. Prices are not given here: check current retailer price. Two 5090s and four 3090s are not covered on this page; see rtx-5090-llm, rtx-3090-llm and multi-gpu-llm.

What PSU do I need for a dual 3090 build?

NVIDIA publishes the number for one card only: 750 W required system power for an RTX 3090 (based on an Intel Core i9-10900K system, and a lower rating “may work depending on system configuration”), 350 W total graphics power, two PCIe 8-pin connectors per card via the included adapter. For two cards the graphics power alone is 2 x 350 = 700 W, before CPU, drives and fans. NVIDIA does not publish a two-card PSU figure, so it is unknown here. Sizing and power-limit detail are on multi-gpu-pcie-and-psu.

What motherboard and case do I need for two GPUs?

The cards are 3-slot per NVIDIA (12.3 inches long; board partner cards vary), so you need two full-length x16 slots with enough spacing for two thick coolers, plus room for an NVLink bridge if you use one. The PCIe lane layout of the second slot is a motherboard spec you read from the board maker, and the speed cost of a lower-lane second slot is unknown here. See multi-gpu-pcie-and-psu.

Questions

Can two RTX 3090s run a 70B model?

Borderline at Q4_K_M with short context. Two cards give 48 GB, the Llama 3.1 70B Q4_K_M file is 43.1 GB per the llama.cpp quantize README, and KV cache is 1.3422 GB at 4,096 tokens by the Qwen2.5-72B stand-in config, for 44.4422 GB before overhead; with the assumed 1 to 2 GB overhead per card that is 46.4422 to 48.4422 GB, so it fits at the low end only. At 32,768 tokens the total is 53.8374 GB before overhead, which does not fit.

Is 48 GB across two 3090s the same as one 48 GB card?

No. The model is split across cards by layer, row or tensor (llama.cpp split modes), and each card keeps its own 24 GB. Capacity adds up for splittable work, but memory is not unified.

Do I need NVLink for two 3090s?

No. NVIDIA lists NVLink (SLI-Ready) on the RTX 3090 and sells a separate bridge, but the speed gain for LLM inference on 3090s is unknown. NVIDIA’s only cited LLM numbers are for H200 servers, with a 1.0x gain at batch 1.

Dual 3090 or a single RTX 5090?

Two 3090s if you need the 43.1 GB 70B Q4_K_M file to fit, since 48 GB is more than the 5090’s 32 GB. The 5090 has no NVLink per NVIDIA and draws 575 W versus 700 W for two 3090s. Speed for either is unknown here.

What PSU do I need for dual 3090?

NVIDIA lists 750 W for one RTX 3090 system and 350 W graphics power per card, so two cards are 700 W for graphics alone. A two-card PSU rating from NVIDIA is not published, so it is unknown.

What motherboard do I need for two GPUs?

Two full-length x16 slots with room for two 3-slot cards. The second slot’s lane count is read from the board maker’s spec, and its effect on LLM speed is unknown here.

Sources read on 2026-10-13

• NVIDIA RTX 3090 / 3090 Ti: 24 GB GDDR6X, 384-bit, NVLink (SLI-Ready), 350 W, 750 W required system power, 2x PCIe 8-pin, 3-slot. Bandwidth not stated.
• NVIDIA RTX 5090: 32 GB GDDR7, 512-bit, 575 W, 1000 W required system power, NVLink “No”. Bandwidth not stated.
• llama.cpp quantize README: Llama 3.1 70B 280.9 GB original, 43.1 GB Q4_K_M.
• llama.cpp CLI README: --split-mode none, layer, row, tensor definitions.
• Qwen2.5-72B model card and config.json: 72.7B parameters, 80 layers, 8 KV heads, 64 attention heads, hidden size 8192, bfloat16. The Llama 3.1 70B config returned HTTP 401.
• NVIDIA NVLink and NVSwitch blog: H200 batch 1 and batch 32 comparison, quoted with the caveat that it is not about 3090s.
• Hacker News item 45300668: title only; no numbers used.

Related

• multi-gpu-llm and multi-gpu-pcie-and-psu cover splitting, lanes and power.
• vram-for-70b-model is the full 70B walk-through.
• rtx-3090-llm and rtx-5090-llm cover the single cards.
• rtx-3090-vs-4090-vs-5090-llm compares three NVIDIA cards.
• vram-requirements and methodology define the formulas.
• mixed-gpus-for-llm covers pairing different cards.