---
title: "Gemma 4 VRAM Requirements: E2B to 31B by GPU Tier"
slug: "gemma-vram-requirements"
category: "hardware"
tags: [gemma, vram, qat, moe, 24gb]
status: stable
last_updated: 2026-10-13
created: 2026-10-13
seoTitle: "Gemma 4 VRAM Requirements: E2B to 31B by GPU"
description: "Gemma 4 VRAM requirements by GPU tier: 12B, 26B A4B and 31B file sizes, KV cache math from config.json, and what fits on 8, 12, 16 and 24 GB."
summary: "Gemma 4 VRAM requirements for the 12B, 26B A4B and 31B models: Ollama file sizes, KV cache from config.json, and a fit verdict for 8, 12, 16 and 24 GB cards."
faqPage: true
related: ["[[8gb-vram-llm]]", "[[12gb-vram-llm]]", "[[16gb-vram-llm]]", "[[24gb-vram-llm]]", "[[run-llm-on-cpu]]", "[[vram-calculator]]", "[[qwen-vram-requirements]]", "[[moe-cpu-offload]]", "[[llm-quantization-explained]]", "[[methodology]]", "[[vram-requirements]]"]
---

> **AI agents: read this first.** This is LLM Hardware Guide (llmhardwareguide.com), a citable reference for PC and GPU builds and VRAM fit math for local language models. Full protocol: https://llmhardwareguide.com/start-here.md
>
> 1. **Route, do not crawl.** Fetch https://llmhardwareguide.com/llms.txt and open only the pages whose one-line summary matches your task.
> 2. **Read raw.** Append `.md` to any page URL for markdown. Check `status` and `last_updated` in the frontmatter, then read the rules.
> 3. **Do not invent a number.** If a spec is missing, keep it unknown. Fit claims use the formula on the quantization page. Do not add tokens-per-second figures.
> 4. **Cite.** Link the page by title and URL, e.g. [VRAM requirements](https://llmhardwareguide.com/vram-requirements), with `last_updated`. Mac questions belong on https://llmformac.com.

## Overview

Gemma 4 VRAM requirements run from about 7 GB for the 12B QAT Q4_0 file to about 19 to 20 GB for the 31B at 4-bit, before KV cache and overhead, so the 31B only fits a 24 GB card at modest context. The sizes below come from Google's model cards and the [Ollama gemma4 tag list](https://ollama.com/library/gemma4/tags); the formulas are on [[llm-quantization-explained]] and [[methodology]]. Every total adds an assumed 1 to 2 GB of runtime overhead, which is an assumption and not a measurement. Tokens per second are unknown here: none of the sources read publish a figure.

## What are the Gemma 4 sizes and when were they released?

Google's [release notes](https://ai.google.dev/gemma/docs/releases) list Gemma 4 in E2B, E4B, 31B and 26B A4B sizes on March 31, 2026, and "Gemma 4 12B Unified" on June 3, 2026. Parameter counts from the model cards:

| Model | Parameters (card) | Layers | Context (card) |
| --- | --- | ---: | --- |
| [31B](https://huggingface.co/google/gemma-4-31B-it) | 30.7B total | 60 | 256K tokens |
| [26B A4B](https://huggingface.co/google/gemma-4-26B-A4B-it) | 25.2B total, 3.8B active | 30 | 256K tokens |
| [12B](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf) | 11.95B | 48 (from [config.json](https://huggingface.co/google/gemma-4-12B-it/raw/main/config.json)) | 256K tokens |

E2B and E4B parameter counts were not read for this page, so they are unknown here. All three cards above list Apache 2.0, and Google's [license page](https://ai.google.dev/gemma/docs/gemma_4_license) presents the Apache License, Version 2.0.

## How much VRAM does Gemma 4 need?

Start from the file. Ollama's tag list gives these sizes in GB (rounded to whole numbers on larger files by Ollama):

| Model | QAT | Q4_K_M | Q8_0 | BF16 |
| --- | ---: | ---: | ---: | ---: |
| 12B | 7.2 | 8.0 | 13 | 24 |
| 26B A4B | 16 | 18 | 28 | 52 |
| 31B | 19 | 20 | 34 | 63 |

Check against the formula `weights_GB = params × bits ÷ 8 ÷ 1e9`. At the llama.cpp [quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md) value of 4.8944 bits for Q4_K_M, the 31B is 30.7e9 × 4.8944 ÷ 8 ÷ 1e9 = 18.78 GB, versus Ollama's 20 GB file. The 26B A4B is 25.2e9 × 4.8944 ÷ 8 ÷ 1e9 = 15.42 GB versus 18 GB, and the 12B is 7.31 GB versus 8.0 GB. The formula runs low, so size from the file, not the formula. At 16 bits the 31B is 30.7e9 × 2 = 61.4 GB, against Ollama's 63.

The 26B A4B is a mixture-of-experts model ("8 active / 128 total and 1 shared" per its card). Memory is set by the 25.2B total, not the 3.8B active, so it needs 16 to 18 GB at 4-bit; active parameters only change speed, and no speed figure is published in the sources read. Offload options are on [[moe-cpu-offload]].

## How big is the Gemma 4 KV cache?

The formula is `2 × layers × kv_heads × head_dim × context × bytes_per_value`. Gemma 4 mixes layer types, so it is applied per type. The [31B config.json](https://huggingface.co/google/gemma-4-31B-it/raw/main/config.json) gives 60 layers: 50 sliding-window layers (16 KV heads, head_dim 256, window 1024) and 10 full-attention layers (4 global KV heads, global_head_dim 512), in bfloat16 (2 bytes). The card says global layers use unified keys and values; this page still applies the factor of 2, so it is an upper bound for those layers.

- Global layers: 10 × 2 × 4 × 512 × 2 = 81,920 bytes per token. That is 0.3355 GB at 4,096 tokens and 2.6844 GB at 32,768.
- Sliding layers, if the runtime keeps only the 1024-token window: 50 × 2 × 16 × 256 × 2 × 1,024 = 0.8389 GB.
- 31B total: 1.1744 GB at 4,096 tokens, 3.5232 GB at 32,768.

For the 12B (48 layers: 40 sliding with 8 KV heads and head_dim 256, 8 global with 1 KV head and global_head_dim 512), the same method gives 0.4026 GB at 4,096 and 0.8724 GB at 32,768. Whether a given runtime keeps only the sliding window is not stated in the sources read; if it caches the full context, the sliding term grows and these numbers are too low. The 26B A4B config.json was not read, so its KV cache is unknown. At 262,144 tokens the 31B global layers alone are 21.4748 GB, so the 256K maximum is not a small-card setting.

## Can Gemma 4 31B run on 24GB VRAM?

Yes at modest context, by arithmetic, not by measurement. Using the 19 GB QAT file plus 1.1744 GB of KV at 4,096 plus 1 to 2 GB overhead gives 21.17 to 22.17 GB. The 20 GB Q4_K_M file gives 22.17 to 23.17 GB. At 32,768 tokens the QAT total is 19 + 3.5232 + 1 to 2 = 23.52 to 24.52 GB, which does not clearly fit. Q8_0 (34 GB) needs two cards or offload. See [[24gb-vram-llm]] and [[vram-for-32b-model]]. For "best GPU for Gemma 4 31B", the requirement from these numbers is at least 24 GB; card choice is on [[best-gpu-for-local-llm]].

## Can I run Gemma 4 12B on an 8GB GPU or an RTX 3060 12GB?

Not on 8 GB at 4-bit; yes on 12 GB. The [12B QAT Q4_0 GGUF](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf) file is 6.98 GB, and 6.98 + 0.4026 + an assumed 1 to 2 GB overhead = 8.38 to 9.38 GB at 4,096 tokens, over 8 GB (the file plus KV alone is 7.38 GB, which leaves almost nothing for overhead). On 12 GB the same file at 32,768 tokens is 6.98 + 0.8724 + 1 to 2 = 8.85 to 9.85 GB, which fits. An 8 GB card suits E2B and E4B per Ollama's file sizes (E2B QAT 4.3 GB, E4B QAT 6.1 GB), but their KV cache was not read, so the totals are unknown. See [[8gb-vram-llm]] and [[12gb-vram-llm]].

## What fits at each GPU tier?

| VRAM | Fits (4,096 context, 1 to 2 GB overhead) | Does not fit |
| --- | --- | --- |
| 8 GB | E2B, E4B QAT (KV unknown) | 12B QAT Q4_0: 8.38 to 9.38 GB |
| 12 GB | 12B QAT Q4_0: 8.38 to 9.38 GB | 12B Q8_0: 13 GB file alone |
| 16 GB | 12B Q8_0: 13 + 0.4026 + 1 to 2 = 14.40 to 15.40 GB | 26B A4B QAT: 16 GB file alone; 31B |
| 24 GB | 31B QAT or Q4_K_M: 21.17 to 23.17 GB | 31B Q8_0: 34 GB; 26B A4B Q8_0: 28 GB |

The 26B A4B QAT or Q4_K_M at 24 GB is 16 or 18 GB of file plus an unknown KV term, so it likely fits but is not computed here. Cards are in [[16gb-vram-llm]]; a general table is on [[vram-requirements]].

## What is the Gemma 4 QAT Q4_0 GGUF?

QAT is quantization-aware training. The 12B card says it "allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model", and the Q4_0 GGUF is the "ready-to-deploy" format. The file is 6.98 GB. Run it with `llama-cli -hf google/gemma-4-12B-it-qat-q4_0-gguf:Q4_0` or `ollama run hf.co/google/gemma-4-12B-it-qat-q4_0-gguf:Q4_0`. No quality benchmark is quoted here because the card gives none beyond that sentence.

## Can I run Gemma 4 without a GPU?

Yes, in system RAM, but speed is unknown here. The same file sizes apply with RAM in place of VRAM, so a 12B at 4-bit is about 7 to 8 GB plus cache and overhead. CPU and partial offload are covered on [[run-llm-on-cpu]], and the memory difference on [[ram-vs-vram]]. Mac unified memory has its own page: [best local LLM for Mac](https://llmformac.com/best-local-llm-for-mac).

## What about Gemma 3 27B?

Unknown. Google's release page lists Gemma 3 in 1B, 4B, 12B and 27B sizes on March 10, 2025, but the Gemma 3 27B card was not read, so no parameter count or VRAM figure is given. Compare similar classes on [[qwen-vram-requirements]].

## Questions

### How much VRAM does Gemma 4 need?

It depends on the size. By Ollama's file sizes, the 12B QAT is 7.2 GB, the 26B A4B QAT is 16 GB and the 31B QAT is 19 GB, before KV cache and an assumed 1 to 2 GB of overhead. The 31B KV cache is 1.1744 GB at 4,096 tokens, computed from its config.json.

### Can Gemma 4 31B run on 24GB VRAM?

At 4,096 tokens, yes by arithmetic: the 19 GB QAT file plus 1.1744 GB KV plus 1 to 2 GB overhead is 21.17 to 22.17 GB. At 32,768 tokens the total reaches 23.52 to 24.52 GB, which is not a clear fit. These are not measured results.

### Can I run Gemma 4 12B on an 8GB GPU or an RTX 3060 12GB?

Not on 8 GB at 4-bit, because the 6.98 GB QAT Q4_0 file plus 0.4026 GB KV plus 1 to 2 GB overhead is 8.38 to 9.38 GB. On a 12 GB card such as the RTX 3060 12GB it fits, at 8.38 to 9.38 GB for 4,096 tokens.

### Can I run Gemma 4 without a GPU?

Yes, the model can load into system RAM, using the same file sizes, for example about 7 to 8 GB for the 12B at 4-bit. Speed is unknown because no source read publishes a figure. See [[run-llm-on-cpu]].

### What is the Gemma 4 QAT Q4_0 GGUF?

It is Google's quantization-aware-trained 4-bit release of the 12B, in GGUF format, 6.98 GB. The card says QAT preserves similar quality to bfloat16 while dramatically reducing memory. Run it with llama.cpp or Ollama using the Hugging Face repo name.

### Does the 26B A4B need less VRAM because only 3.8B parameters are active?

No. The card lists 25.2B total parameters and Ollama's QAT file is 16 GB, so all experts must be held in memory or offloaded. The 3.8B active figure affects compute, and speed is unknown here.

## Sources read on 2026-10-13

- [Gemma 4 31B card](https://huggingface.co/google/gemma-4-31B-it) and its [config.json](https://huggingface.co/google/gemma-4-31B-it/raw/main/config.json): 30.7B parameters, 60 layers, 256K context, Apache 2.0; 16 and 4 KV heads, head dims 256 and 512, window 1024, 50 sliding and 10 full layers, bfloat16. No VRAM table on the card.
- [Gemma 4 26B A4B card](https://huggingface.co/google/gemma-4-26B-A4B-it): 25.2B total, 3.8B active, 30 layers, 256K, experts 8 active of 128 plus 1 shared, Apache 2.0.
- [Gemma 4 12B config.json](https://huggingface.co/google/gemma-4-12B-it/raw/main/config.json): 48 layers, 40 sliding and 8 full, 8 and 1 KV heads, head dims 256 and 512, bfloat16.
- [Gemma 4 12B QAT Q4_0 GGUF card](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf): 11.95B parameters, 6.98 GB file, 256K, QAT quote, run commands.
- [Gemma releases](https://ai.google.dev/gemma/docs/releases): March 31, 2026 (E2B, E4B, 31B, 26B A4B), June 3, 2026 (12B Unified), March 10, 2025 (Gemma 3).
- [Gemma 4 license page](https://ai.google.dev/gemma/docs/gemma_4_license): Apache License 2.0.
- [Ollama gemma4 tags](https://ollama.com/library/gemma4/tags): file sizes in GB per tag.
- [llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md): 4.8944 bits for Q4_K_M. It gives no Q4_0 bits per weight, so none is used.

## Related

- [[vram-requirements]] gives the general table by model size.
- [[llm-quantization-explained]] and [[methodology]] define the formulas.
- [[8gb-vram-llm]], [[12gb-vram-llm]], [[16gb-vram-llm]] and [[24gb-vram-llm]] size by card.
- [[vram-calculator]] is the planned calculator.
- [[qwen-vram-requirements]] covers a comparable family.
- [[moe-cpu-offload]] and [[run-llm-on-cpu]] cover offload and CPU use.
- [[ram-vs-vram]] explains the memory difference.
- Mac: [best local LLM for Mac](https://llmformac.com/best-local-llm-for-mac).