Overview
The RTX 3090 and RTX 4090 are both 24 GB cards on NVIDIA’s pages. The RTX 5090 is a 32 GB card. For a local LLM, that capacity gap matters more than the generation name. Fit arithmetic is llm-quantization-explained and vram-requirements.
Which is better for local LLMs: RTX 3090, 4090 or 5090?
On VRAM, the 5090 holds more bytes: 32 GB GDDR7 versus 24 GB GDDR6X on both the 3090 and the 4090. This page does not call any of the three faster. NVIDIA’s pages, read on 2026-10-12, do not publish an LLM tokens-per-second result, and memory bandwidth is not listed on the fetched 3090, 4090, or 5090 spec tables, so bandwidth is unknown for all three.
| Spec | RTX 3090 | RTX 4090 | RTX 5090 |
|---|---|---|---|
| Memory | 24 GB GDDR6X | 24 GB GDDR6X | 32 GB GDDR7 |
| Interface | 384-bit | 384-bit | 512-bit |
| CUDA cores | 10496 | 16384 | 21760 |
| Boost clock | 1.70 GHz | 2.52 GHz | 2.41 GHz |
| Card power | 350 W graphics card power | 450 W total graphics power | 575 W total graphics power |
| Required system power | 750 W | 850 W | 1000 W |
| NVLink | Yes | No | No |
| Length / slot | 313 mm, 3-slot | 304 mm, 3-slot | 304 mm, 2-slot |
| Memory bandwidth | unknown on the fetched page | unknown on the fetched page | unknown on the fetched page |
Sources: 3090 family (3090 column, not 3090 Ti), 4090, 5090. The 3090 Ti column on the same family page is a different card (10752 CUDA cores, 1.86 GHz boost, 450 W graphics card power, 850 W required system power) and is not the 3090 in this comparison.
Is a 4090 worth it over a 3090 for LLMs?
They list the same 24 GB capacity and the same 384-bit interface. A model that does not fit in 24 GB fits on neither. What the pages do show is a higher total graphics power on the 4090 (450 W versus 350 W graphics card power on the 3090), a higher required system power (850 W versus 750 W), more CUDA cores (16384 versus 10496), and NVLink on the 3090 only. Whether that trade is worth the money depends on what the cards cost. Check current retailer price. 24gb-vs-16gb-vs-12gb-vram shows what 24 GB holds.
What models fit on a single RTX 5090?
32 GB is the budget. Qwen2.5-32B Q4_K_M weights are 19.8835 decimal GB. KV at 4,096 tokens is 1.0737 GB, and at 32,768 tokens it is 8.5899 GB. With an assumed 1 to 2 GB of overhead, the 32,768-token total is 29.4734 to 30.4734 GB, under 32 GB. At the card’s kind of long context, Qwen2.5-32B’s 131,072-token KV term alone is 34.3597 GB, which is over 32 GB before weights.
Qwen2.5-32B Q8_0 weights are 34.5345 GB, over 32 GB before KV. Llama 3.1 70B Q4_K_M is a 43.1 GB file. Qwen2.5-72B Q4_K_M weights estimate to 44.4779 GB. The 5090 does not hold those 70B-class Q4_K_M weights on one card. It is a 32 GB card, not a 70B Q4 card.
What power supply do these cards need?
Quote the required system power, including the footnote CPU, and do not collapse two lines into one.
- RTX 3090: 750 W, based on a Core i9-10900K. NVIDIA says a lower rating may work depending on configuration. Graphics card power is 350 W.
- RTX 4090: 850 W, based on a Ryzen 9 5900X. Total graphics power is 450 W. The prepare section also says “850W Minimum” and that a higher rating may be required.
- RTX 5090: 1000 W, based on a Ryzen 9 9950X, with a note to prefer a PCIe CEM 5.1 power supply. Total graphics power is 575 W. The prepare section separately says “850W Minimum” and that a higher rating may be required. Those are both on the page. This guide does not pick one of them as the only number.
local-llm-pc-build uses these power figures for the rest of the machine. A Mac comparison is which Mac for a local LLM.
Sources read on 2026-10-12
- llama.cpp quantize README for bits per weight, GiB sizes, and the Memory/Disk table.
- Qwen2.5-7B, 14B, 32B, and 72B model cards, plus each repo’s
config.json, for parameter counts, layers, heads, hidden size, andtorch_dtype. - RTX 5090, RTX 4090, RTX 3090 family, and RTX 4060 family.
- Radeon RX 7900 XTX and Radeon RX 9070 XT.
- Ollama GPU, Ollama FAQ, Ollama API, and llama.cpp CLI options.
- LM Studio system requirements.
- Intel Arc B580 specifications at
https://www.intel.com/content/www/us/en/products/sku/241598/intel-arc-b580-graphics/specifications.htmlreturned HTTP 403. No Intel VRAM figure is cited.
Related
- llm-quantization-explained is the formula.
- vram-requirements applies it by model size.
- 24gb-vs-16gb-vs-12gb-vram applies it by card size.
- best-gpu-for-local-llm names the cards.
- rtx-3090-vs-4090-vs-5090-llm compares three NVIDIA cards.
- local-llm-pc-build covers the rest of the PC.
- run-llm-on-cpu covers CPU and partial offload.
- methodology is the citation rule.
- Mac memory questions: what fits my RAM and how much RAM for a local LLM.