Best LLM for 8GB VRAM: What Fits and What Doesn't

Overview

The best LLM for 8GB VRAM is a 7B to 8B model at Q4_K_M with a modest context, because a 7.61B model needs 5.8907 to 6.8907 GB at 4,096 tokens by the formula below, while a 12B model’s file alone is 6.98 GB and leaves about 1.02 GB for cache and overhead. This page shows the arithmetic with the site’s methodology formulas so you can check any model against an 8 GB card. Tokens per second are unknown here: no vendor page or README read for this page publishes a figure with a named machine.

Which cards have 8GB of VRAM?

Per NVIDIA’s own pages (read 2026-10-13): the RTX 5060 family page lists the RTX 5060 at 8 GB GDDR7 and the RTX 5060 Ti as “16 GB / 8 GB GDDR7”; the RTX 4060 family page lists the RTX 4060 at 8 GB GDDR6 and the 4060 Ti as “16 GB GDDR6 or 8 GB GDDR6”; the RTX 3070 family page lists the RTX 3070 at 8 GB GDDR6 and the 3070 Ti at 8 GB GDDR6X. The 5060 Ti and 4060 Ti are sold in two sizes, so check which one you have with how-to-check-vram. Memory bandwidth is unknown: those pages list bus width (128-bit on the 5060 and 4060 families), not bandwidth. Prices: check current retailer price.

What can you run on 8GB VRAM?

Weights are params × bits_per_weight ÷ 8 ÷ 1e9 GB. KV cache is 2 × layers × kv_heads × head_dim × context × 2 bytes (bfloat16). Overhead is an assumed 1 to 2 GB, not a measurement (llm-quantization-explained). Bits per weight are the llama.cpp quantize README Q4_K_M value 4.8944; parameters, layers and heads are from each Qwen2.5 model card and config.json. Context is 4,096 tokens, decimal GB.

Rows are examples of size classes, not a ranking of quality; quality is not measured on this page. Qwen2.5 is used because its layer and head counts were read from config.json. A Llama 3.1 8B Q4_K_M file is 4.9 GB per the same README, but its KV term is unknown here.

Can an 8GB GPU run a 12B model?

Not fully on the GPU at a useful context. Google’s Gemma 4 12B QAT Q4_0 GGUF is 6,975,879,296 bytes (6.9759 GB; the card lists 6.98 GB), per the repository file listing. That leaves 8 − 6.9759 = 1.0241 GB before any KV cache or overhead, which is about the size of the 1 to 2 GB overhead assumption alone, before any cache. The card gives 48 layers and a 1,024-token sliding window; its config.json gives 8 KV heads of dimension 256. The simple KV formula (2 × 48 × 8 × 256 × 4,096 × 2 bytes = 1.6106 GB) treats every layer as full attention, so it is an upper bound for this hybrid model. Applying the per-layer-type method on gemma-vram-requirements (40 sliding layers capped at the 1,024-token window with 8 KV heads of head dimension 256, 8 global layers with 1 KV head of head dimension 512) gives 0.4026 GB. Which one a runtime allocates is unknown here, and the file plus either KV figure plus the assumed 1 to 2 GB overhead is over 8 GB. Part of the model can be placed in system RAM, which is slower; see run-llm-on-cpu and gemma-vram-requirements. The 12B class is a fit on the next tier up: 12gb-vram-llm.

What about gpt-oss-20b on 8GB or 6GB?

The model card says gpt-oss-20b has 21B parameters with 3.6B active and that it can “run within 16GB of memory”, with MXFP4 quantization of the MoE weights. The repository’s original/model.safetensors is 13,761,300,984 bytes, 13.7613 GB, so the full model cannot sit in 8 GB or 6 GB of VRAM. Because it is a mixture-of-experts model, only part of the weights are active per token, which is why offloading experts to system RAM is discussed in moe-cpu-offload. How fast that runs on an 8 GB card is unknown. A “6 GB” setup is the same story with 2 GB less room.

How much context fits in 8GB of VRAM?

Solve the formula for context. For Qwen2.5-7B the KV cost is 2 × 28 × 4 × 128 × 2 = 57,344 bytes per token. With 8 − 4.6558 − overhead GB free, that is 2.3442 GB ÷ 57,344 B ≈ 40,880 tokens at 1 GB overhead and 1.3442 GB ÷ 57,344 B ≈ 23,441 tokens at 2 GB overhead. The card’s maximum is 131,072 tokens, so the full window does not fit. Treat these as formula results; runtimes may quantize the cache or allocate differently, and measured limits are unknown. A larger model shrinks the number fast: Qwen2.5-14B does not fit at all, since its 8.9935 GB of weights alone exceed 8 GB.

What is the best local model for 4GB or 6GB VRAM?

A 3B-class model at Q4, as the table shows: Qwen2.5-3B totals 3.0415 to 4.0415 GB at 4,096 tokens, so it is tight on 4 GB and comfortable on 6 GB, and a 7B at 5.8907 to 6.8907 GB is not safe on 6 GB. For laptops, NVIDIA’s laptop compare page lists the RTX 4050 Laptop GPU at 6 GB GDDR6 and the RTX 4060 Laptop GPU at 8 GB GDDR6 (read by column position, as the page does not label each cell). It lists no 4 GB laptop GPU in the 40 Series section; 4 GB card VRAM is otherwise unknown here. Laptop VRAM varies by model, so confirm yours first. The Gemma 4 card also names smaller E2B and E4B sizes; their file sizes were not read, so they are unknown here.

Is there a better use of the money than 8GB?

The jump that changes what fits is 12 GB or 16 GB, see 24gb-vs-16gb-vs-12gb-vram and budget-gpu-for-local-llm. If the model does not fit, the error is covered in cuda-out-of-memory-llm. On a Mac, memory is unified; see best local LLM for Mac.

Questions

What can you run on 8GB VRAM?

Models whose weights, KV cache and 1 to 2 GB of overhead total under 8 GB, which in practice means 3B to 8B models at Q4. Qwen2.5-7B at Q4_K_M and 4,096 tokens is 5.8907 to 6.8907 GB by the formula; a 12B file alone is 6.98 GB and a 20B is 13.76 GB.

What models can I actually run at a decent context locally on 8 GB VRAM?

A 7B-class model leaves 1.3442 to 2.3442 GB for KV cache after 4.6558 GB of weights and the overhead assumption, which the formula turns into about 23,000 to 40,900 tokens for Qwen2.5-7B. Measured limits are unknown; the figure is arithmetic from the model card’s config.json.

What is the best local model for 4GB or 6GB VRAM?

A 3B model at Q4. Qwen2.5-3B is 1.8905 GB of weights and 3.0415 to 4.0415 GB total at 4,096 tokens. NVIDIA’s laptop compare page lists the RTX 4050 Laptop GPU at 6 GB; no 4 GB laptop GPU is listed there, so 4 GB laptop VRAM is unknown.

Can an 8GB GPU run a 12B model?

Not entirely on the GPU. The Gemma 4 12B QAT Q4_0 GGUF is 6.98 GB on its card, leaving about 1.02 GB for cache and overhead. Offloading some layers to system RAM works but is slower, and the speed is unknown.

How much context fits in 8GB of VRAM?

For Qwen2.5-7B at Q4_K_M, about 23,441 to 40,880 tokens by the KV formula, depending on whether overhead is 2 or 1 GB. Models with more KV heads or layers fit less; Qwen2.5-14B does not fit at all.

Sources read on 2026-10-13

• NVIDIA RTX 5060 family: RTX 5060 8 GB GDDR7; RTX 5060 Ti “16 GB / 8 GB GDDR7”; 128-bit; bandwidth not listed.
• NVIDIA RTX 4060 family: RTX 4060 8 GB GDDR6; 4060 Ti “16 GB GDDR6 or 8 GB GDDR6”; 128-bit.
• NVIDIA RTX 3070 family: 3070 8 GB GDDR6; 3070 Ti 8 GB GDDR6X.
• NVIDIA laptop compare: RTX 4050 Laptop 6 GB GDDR6, RTX 4060 Laptop 8 GB GDDR6 (positional reading); no 4 GB laptop GPU in the 40 Series section.
• Gemma 4 12B QAT Q4_0 GGUF card and its file listing: 6,975,879,296-byte GGUF, 48 layers, 1,024 sliding window, 256K context; plus gemma-4-12B-it config.json: 8 KV heads, head dim 256.
• gpt-oss-20b card, its config.json (24 layers, 8 KV heads, head dim 64) and file listing (13,761,300,984-byte original/model.safetensors): 21B total, 3.6B active, MXFP4, “within 16GB”.
• Qwen2.5-3B, 7B and 14B cards and config.json: parameters 3.09B, 7.61B, 14.7B; layers 36, 28, 48; KV heads 2, 4, 8; bfloat16.
• llama.cpp quantize README: Q4_K_M 4.8944 bits per weight; Llama 3.1 8B Q4_K_M 4.9 GB.
• Ahrefs US search volumes pulled 2026-10-12 are internal research and not cited on this page.

Related

• vram-requirements is the size-by-model table.
• 12gb-vram-llm and 24gb-vs-16gb-vs-12gb-vram cover the next tiers.
• llm-quantization-explained and methodology are the formulas.
• gemma-vram-requirements, moe-cpu-offload and run-llm-on-cpu cover models that do not fit.
• vram-calculator, gpu-vram-list, how-to-check-vram, cuda-out-of-memory-llm, budget-gpu-for-local-llm.