Gemma 4 VRAM Requirements: E2B to 31B by GPU Tier

Overview

Gemma 4 VRAM requirements run from about 7 GB for the 12B QAT Q4_0 file to about 19 to 20 GB for the 31B at 4-bit, before KV cache and overhead, so the 31B only fits a 24 GB card at modest context. The sizes below come from Google’s model cards and the Ollama gemma4 tag list; the formulas are on llm-quantization-explained and methodology. Every total adds an assumed 1 to 2 GB of runtime overhead, which is an assumption and not a measurement. Tokens per second are unknown here: none of the sources read publish a figure.

What are the Gemma 4 sizes and when were they released?

Google’s release notes list Gemma 4 in E2B, E4B, 31B and 26B A4B sizes on March 31, 2026, and “Gemma 4 12B Unified” on June 3, 2026. Parameter counts from the model cards:

E2B and E4B parameter counts were not read for this page, so they are unknown here. All three cards above list Apache 2.0, and Google’s license page presents the Apache License, Version 2.0.

How much VRAM does Gemma 4 need?

Start from the file. Ollama’s tag list gives these sizes in GB (rounded to whole numbers on larger files by Ollama):

Check against the formula weights_GB = params × bits ÷ 8 ÷ 1e9. At the llama.cpp quantize README value of 4.8944 bits for Q4_K_M, the 31B is 30.7e9 × 4.8944 ÷ 8 ÷ 1e9 = 18.78 GB, versus Ollama’s 20 GB file. The 26B A4B is 25.2e9 × 4.8944 ÷ 8 ÷ 1e9 = 15.42 GB versus 18 GB, and the 12B is 7.31 GB versus 8.0 GB. The formula runs low, so size from the file, not the formula. At 16 bits the 31B is 30.7e9 × 2 = 61.4 GB, against Ollama’s 63.

The 26B A4B is a mixture-of-experts model (“8 active / 128 total and 1 shared” per its card). Memory is set by the 25.2B total, not the 3.8B active, so it needs 16 to 18 GB at 4-bit; active parameters only change speed, and no speed figure is published in the sources read. Offload options are on moe-cpu-offload.

How big is the Gemma 4 KV cache?

The formula is 2 × layers × kv_heads × head_dim × context × bytes_per_value. Gemma 4 mixes layer types, so it is applied per type. The 31B config.json gives 60 layers: 50 sliding-window layers (16 KV heads, head_dim 256, window 1024) and 10 full-attention layers (4 global KV heads, global_head_dim 512), in bfloat16 (2 bytes). The card says global layers use unified keys and values; this page still applies the factor of 2, so it is an upper bound for those layers.

• Global layers: 10 × 2 × 4 × 512 × 2 = 81,920 bytes per token. That is 0.3355 GB at 4,096 tokens and 2.6844 GB at 32,768.
• Sliding layers, if the runtime keeps only the 1024-token window: 50 × 2 × 16 × 256 × 2 × 1,024 = 0.8389 GB.
• 31B total: 1.1744 GB at 4,096 tokens, 3.5232 GB at 32,768.

For the 12B (48 layers: 40 sliding with 8 KV heads and head_dim 256, 8 global with 1 KV head and global_head_dim 512), the same method gives 0.4026 GB at 4,096 and 0.8724 GB at 32,768. Whether a given runtime keeps only the sliding window is not stated in the sources read; if it caches the full context, the sliding term grows and these numbers are too low. The 26B A4B config.json was not read, so its KV cache is unknown. At 262,144 tokens the 31B global layers alone are 21.4748 GB, so the 256K maximum is not a small-card setting.

Can Gemma 4 31B run on 24GB VRAM?

Yes at modest context, by arithmetic, not by measurement. Using the 19 GB QAT file plus 1.1744 GB of KV at 4,096 plus 1 to 2 GB overhead gives 21.17 to 22.17 GB. The 20 GB Q4_K_M file gives 22.17 to 23.17 GB. At 32,768 tokens the QAT total is 19 + 3.5232 + 1 to 2 = 23.52 to 24.52 GB, which does not clearly fit. Q8_0 (34 GB) needs two cards or offload. See 24gb-vram-llm and vram-for-32b-model. For “best GPU for Gemma 4 31B”, the requirement from these numbers is at least 24 GB; card choice is on best-gpu-for-local-llm.

Can I run Gemma 4 12B on an 8GB GPU or an RTX 3060 12GB?

Not on 8 GB at 4-bit; yes on 12 GB. The 12B QAT Q4_0 GGUF file is 6.98 GB, and 6.98 + 0.4026 + an assumed 1 to 2 GB overhead = 8.38 to 9.38 GB at 4,096 tokens, over 8 GB (the file plus KV alone is 7.38 GB, which leaves almost nothing for overhead). On 12 GB the same file at 32,768 tokens is 6.98 + 0.8724 + 1 to 2 = 8.85 to 9.85 GB, which fits. An 8 GB card suits E2B and E4B per Ollama’s file sizes (E2B QAT 4.3 GB, E4B QAT 6.1 GB), but their KV cache was not read, so the totals are unknown. See 8gb-vram-llm and 12gb-vram-llm.

What fits at each GPU tier?

The 26B A4B QAT or Q4_K_M at 24 GB is 16 or 18 GB of file plus an unknown KV term, so it likely fits but is not computed here. Cards are in 16gb-vram-llm; a general table is on vram-requirements.

What is the Gemma 4 QAT Q4_0 GGUF?

QAT is quantization-aware training. The 12B card says it “allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model”, and the Q4_0 GGUF is the “ready-to-deploy” format. The file is 6.98 GB. Run it with llama-cli -hf google/gemma-4-12B-it-qat-q4_0-gguf:Q4_0 or ollama run hf.co/google/gemma-4-12B-it-qat-q4_0-gguf:Q4_0. No quality benchmark is quoted here because the card gives none beyond that sentence.

Can I run Gemma 4 without a GPU?

Yes, in system RAM, but speed is unknown here. The same file sizes apply with RAM in place of VRAM, so a 12B at 4-bit is about 7 to 8 GB plus cache and overhead. CPU and partial offload are covered on run-llm-on-cpu, and the memory difference on ram-vs-vram. Mac unified memory has its own page: best local LLM for Mac.

What about Gemma 3 27B?

Unknown. Google’s release page lists Gemma 3 in 1B, 4B, 12B and 27B sizes on March 10, 2025, but the Gemma 3 27B card was not read, so no parameter count or VRAM figure is given. Compare similar classes on qwen-vram-requirements.

Questions

How much VRAM does Gemma 4 need?

It depends on the size. By Ollama’s file sizes, the 12B QAT is 7.2 GB, the 26B A4B QAT is 16 GB and the 31B QAT is 19 GB, before KV cache and an assumed 1 to 2 GB of overhead. The 31B KV cache is 1.1744 GB at 4,096 tokens, computed from its config.json.

Can Gemma 4 31B run on 24GB VRAM?

At 4,096 tokens, yes by arithmetic: the 19 GB QAT file plus 1.1744 GB KV plus 1 to 2 GB overhead is 21.17 to 22.17 GB. At 32,768 tokens the total reaches 23.52 to 24.52 GB, which is not a clear fit. These are not measured results.

Can I run Gemma 4 12B on an 8GB GPU or an RTX 3060 12GB?

Not on 8 GB at 4-bit, because the 6.98 GB QAT Q4_0 file plus 0.4026 GB KV plus 1 to 2 GB overhead is 8.38 to 9.38 GB. On a 12 GB card such as the RTX 3060 12GB it fits, at 8.38 to 9.38 GB for 4,096 tokens.

Can I run Gemma 4 without a GPU?

Yes, the model can load into system RAM, using the same file sizes, for example about 7 to 8 GB for the 12B at 4-bit. Speed is unknown because no source read publishes a figure. See run-llm-on-cpu.

What is the Gemma 4 QAT Q4_0 GGUF?

It is Google’s quantization-aware-trained 4-bit release of the 12B, in GGUF format, 6.98 GB. The card says QAT preserves similar quality to bfloat16 while dramatically reducing memory. Run it with llama.cpp or Ollama using the Hugging Face repo name.

Does the 26B A4B need less VRAM because only 3.8B parameters are active?

No. The card lists 25.2B total parameters and Ollama’s QAT file is 16 GB, so all experts must be held in memory or offloaded. The 3.8B active figure affects compute, and speed is unknown here.

Sources read on 2026-10-13

• Gemma 4 31B card and its config.json: 30.7B parameters, 60 layers, 256K context, Apache 2.0; 16 and 4 KV heads, head dims 256 and 512, window 1024, 50 sliding and 10 full layers, bfloat16. No VRAM table on the card.
• Gemma 4 26B A4B card: 25.2B total, 3.8B active, 30 layers, 256K, experts 8 active of 128 plus 1 shared, Apache 2.0.
• Gemma 4 12B config.json: 48 layers, 40 sliding and 8 full, 8 and 1 KV heads, head dims 256 and 512, bfloat16.
• Gemma 4 12B QAT Q4_0 GGUF card: 11.95B parameters, 6.98 GB file, 256K, QAT quote, run commands.
• Gemma releases: March 31, 2026 (E2B, E4B, 31B, 26B A4B), June 3, 2026 (12B Unified), March 10, 2025 (Gemma 3).
• Gemma 4 license page: Apache License 2.0.
• Ollama gemma4 tags: file sizes in GB per tag.
• llama.cpp quantize README: 4.8944 bits for Q4_K_M. It gives no Q4_0 bits per weight, so none is used.

Related

• vram-requirements gives the general table by model size.
• llm-quantization-explained and methodology define the formulas.
• 8gb-vram-llm, 12gb-vram-llm, 16gb-vram-llm and 24gb-vram-llm size by card.
• vram-calculator is the planned calculator.
• qwen-vram-requirements covers a comparable family.
• moe-cpu-offload and run-llm-on-cpu cover offload and CPU use.
• ram-vs-vram explains the memory difference.
• Mac: best local LLM for Mac.