Best GPU for Ollama: VRAM and Model Size Guide

Overview

The best GPU for Ollama is the one with enough VRAM to hold the model tag you want to run, because Ollama’s own docs say that with less VRAM it falls back to system RAM and “responses may be slower”. Ollama’s quickstart recommends 8 GB of available VRAM (or Mac unified memory), and every step up in model size is a step up in the VRAM you need. This page maps Ollama library tag sizes to 8, 16, 24 and 32 GB cards. It has no benchmarks: no tokens-per-second figure is read from a vendor page for these cards, so speed is unknown here. Install steps live in the Ollama quickstart; this is not a download page. Mac users: LM Studio vs Ollama on Mac.

What is the best GPU for Ollama?

Choose by VRAM first, brand second. Ollama runs on NVIDIA, AMD and Apple GPUs, so no brand is required. The fit test is: file size of the tag, plus KV cache for your context, plus overhead, must be under the card’s VRAM. The method and formulas are on methodology and vram-requirements.

Card sizes below come from NVIDIA’s spec pages: RTX 4060 is 8 GB GDDR6, RTX 4060 Ti is sold as 16 GB or 8 GB, RTX 3090 is 24 GB GDDR6X, and RTX 5090 is 32 GB GDDR7. These are examples of each VRAM tier, not a ranking. Prices are not given: check current retailer price.

Sizes are from the gemma4 tags and qwen3.8 tags pages on 2026-10-13. Those pages print “GB” and do not say whether that is decimal or binary, so treat each as approximate.

How much VRAM does Ollama need? Ollama VRAM requirements by model

Ollama VRAM requirements are the tag size shown in the library plus KV cache plus overhead. Headroom is simple subtraction: headroom = card VRAM minus tag size.

Headroom must also cover the KV cache and overhead. This site assumes 1 to 2 GB of overhead; that is an assumption, not a measurement. So 1.4 GB on an 8 GB card and 2 GB on a 32 GB card are borderline, not fits. The KV cache for these Gemma and Qwen tags is unknown on this page because their config.json files were not read. The method for a model whose config you do read is KV bytes = 2 x layers x kv_heads x head_dim x context x bytes_per_value, worked in vram-requirements.

Context matters. The Ollama FAQ says “By default, Ollama uses a context window size of 4096 tokens”, while the library lists windows of 128K to 256K for these tags. Raising context raises KV memory. The FAQ also documents OLLAMA_KV_CACHE_TYPE: default f16, q8_0 uses about half the memory of f16, and q4_0 about a quarter, with Flash Attention enabled.

What is the minimum GPU for Ollama?

For Ollama’s GPU rules, the minimum is a supported GPU from Ollama’s GPU docs. NVIDIA needs compute capability 5.0 or higher with driver 550 or newer, and cards at 5.0 through 6.2 need driver 570 or newer. AMD needs the ROCm v7 driver on Linux, and a ROCm v7 / HIP7-capable driver stack on Windows. Vulkan is enabled by default when the backend is installed (the docs do not name operating systems in that line). Apple devices use Metal.

For a useful minimum, Ollama’s quickstart recommends 8 GB of available VRAM or unified memory; its local example is gemma4:e2b, and it says “The model download is about 7.2 GB” without naming a tag. In the library, gemma4:e2b-it-q4_K_M is 4.6 GB. A smaller card runs smaller tags, or spills to system RAM at lower speed. Whether a specific GPU is on the support list is unknown to this page for any card not named in those docs: check the docs.

What is the cheapest GPU for Ollama?

Unknown: this site does not publish street prices. The cheapest card that works is the lowest-VRAM card that meets the minimum above and holds your tag with 1 to 2 GB of headroom. Check current retailer price, and compare used and new options on budget-gpu-for-local-llm. A card with less VRAM than the tag forces system RAM offload; see ram-vs-vram.

What are the Ollama GPU requirements and system requirements?

Ollama GPU requirements are the driver and compute-capability rules above. The quickstart gives no minimum system RAM, CPU requirement or list of named GPUs, so those figures are unknown here. It does say macOS, Windows and Linux are supported. To confirm a model loaded onto your GPU, run ollama ps: the FAQ says “100% GPU” means the model loaded entirely into the GPU, and other values mean CPU-only or a CPU/GPU split. If it is not using the GPU, see ollama-not-using-gpu.

Is NVIDIA or AMD better for Ollama?

Both are supported, so the sourced answer is VRAM per card, not brand. NVIDIA’s requirement is compute capability 5.0+; AMD’s is ROCm v7. AMD’s RX 9070 XT page lists 16 GB GDDR6 (read 2026-10-13 after an initial HTTP 503). See amd-gpu-for-local-llm. If you serve many users rather than one, compare runtimes at llama-cpp-vs-ollama-vs-vllm.

Questions

What is the best GPU for Ollama?

The one with enough VRAM for your tag. On a 24 GB card such as the RTX 3090 (24 GB GDDR6X per NVIDIA), gemma4:31b-it-q4_K_M at 20 GB leaves 4 GB before KV cache and overhead; on a 32 GB RTX 5090 it leaves 12 GB. Speed is unknown because no vendor figure is cited.

What is the minimum GPU for Ollama?

An NVIDIA card with compute capability 5.0 or higher and driver 550 or newer, or a supported AMD GPU on ROCm v7, or an Apple device with Metal. For practical use, Ollama’s quickstart recommends 8 GB of available VRAM or unified memory.

What is the cheapest GPU for Ollama?

Unknown, because prices change and this page publishes none. Pick the lowest-VRAM supported card that fits your tag plus 1 to 2 GB of assumed overhead, and check current retailer price.

How much VRAM does Ollama need?

It needs the model tag’s size plus KV cache plus overhead. Ollama’s quickstart recommends 8 GB of available VRAM or Mac unified memory. In the library, gemma4:12b-it-q4_K_M is 8.0 GB, gemma4:26b-a4b-it-q4_K_M is 18 GB and gemma4:31b-it-q4_K_M is 20 GB.

What are the Ollama GPU requirements?

NVIDIA compute capability 5.0+ with driver 550+ (570+ for capability 5.0 to 6.2), AMD ROCm v7 on Linux or a ROCm v7 / HIP7-capable stack on Windows, Vulkan (enabled by default when the backend is installed), or Metal on Apple devices, per the Ollama GPU docs.

Can Ollama run without a GPU?

Yes. The quickstart says that with less VRAM Ollama can use system RAM, but responses may be slower. How much slower is unknown here. See run-llm-on-cpu.

Can I reduce Ollama’s memory use?

Yes, in part. The FAQ documents OLLAMA_KV_CACHE_TYPE=q8_0 for about half the KV memory of f16, or q4_0 for about a quarter with a small-to-medium precision loss, when Flash Attention is on. It does not shrink the weights.

Sources read on 2026-10-13

• Ollama quickstart: 8 GB VRAM recommendation, 7.2 GB model download (local example gemma4:e2b), RAM fallback wording, no stated system RAM or CPU minimum.
• Ollama GPU docs: NVIDIA compute capability and driver versions, AMD ROCm v7, Vulkan, Metal.
• Ollama FAQ: 4096 default context, ollama ps, Flash Attention, OLLAMA_KV_CACHE_TYPE options.
• Ollama library: model families and parameter sizes.
• gemma4 tags and qwen3.8 tags: tag sizes and context windows.
• NVIDIA: RTX 5090 (32 GB GDDR7), RTX 3090 and 3090 Ti (24 GB GDDR6X), RTX 4060 and 4060 Ti (8 GB; 16 GB or 8 GB).
• llama.cpp quantize README: read for the formula context only; no figure from it is used on this page.
• AMD RX 9070 XT: 16 GB GDDR6, 256-bit, up to 640 GB/s; read 2026-10-13 after two HTTP 503 responses.

Related

• best-gpu-for-local-llm is the broader GPU pick.
• ollama-not-using-gpu fixes a model running on CPU.
• budget-gpu-for-local-llm covers the cheap end.
• 16gb-vram-llm and 24gb-vram-llm go deep on two tiers.
• vram-requirements has the fit formula and methodology the citation rule.
• ram-vs-vram, how-to-check-vram and amd-gpu-for-local-llm cover memory types, checking and AMD.
• llama-cpp-vs-ollama-vs-vllm compares runtimes. Mac: LM Studio vs Ollama on Mac.