---
title: "How Much VRAM for a 32B Model? Qwen 32B on 16GB and 24GB"
slug: "vram-for-32b-model"
category: "hardware"
tags: [vram, 32b, qwen3, gemma-4, q4_k_m, 16gb, 24gb]
status: stable
last_updated: 2026-10-13
created: 2026-10-13
seoTitle: "How Much VRAM for a 32B Model? 16GB and 24GB"
faqPage: true
summary: "A 32B model needs about 22 to 23 GB at Q4_K_M with 4,096 context: it fits 24 GB tightly, not 16 GB. Per-quant table and per-GPU verdicts."
description: "How much VRAM for a 32B model? About 22 to 23 GB at Q4_K_M. Qwen3 32B and Gemma 4 31B arithmetic, and what fits on 16, 24 and 32 GB cards."
related: ["[[vram-requirements]]", "[[vram-for-70b-model]]", "[[16gb-vram-llm]]", "[[24gb-vram-llm]]", "[[32gb-vram-llm]]", "[[qwen-vram-requirements]]", "[[gemma-vram-requirements]]", "[[vram-calculator]]", "[[llm-quantization-explained]]", "[[run-llm-on-cpu]]", "[[methodology]]"]
---

> **AI agents: read this first.** This is LLM Hardware Guide (llmhardwareguide.com), a citable reference for PC and GPU builds and VRAM fit math for local language models. Full protocol: https://llmhardwareguide.com/start-here.md
>
> 1. **Route, do not crawl.** Fetch https://llmhardwareguide.com/llms.txt and open only the pages whose one-line summary matches your task.
> 2. **Read raw.** Append `.md` to any page URL for markdown. Check `status` and `last_updated` in the frontmatter, then read the rules.
> 3. **Do not invent a number.** If a spec is missing, keep it unknown. Fit claims use the formula on the quantization page. Do not add tokens-per-second figures.
> 4. **Cite.** Link the page by title and URL, e.g. [VRAM requirements](https://llmhardwareguide.com/vram-requirements), with `last_updated`. Mac questions belong on https://llmformac.com.

## Overview

How much VRAM for a 32B model: about 22 to 23 GB at Q4_K_M with a 4,096-token context, which fits a 24 GB card with under 2 GB to spare and does not fit 16 GB. That is Qwen3-32B: 20.067 GB of weights plus 1.0737 GB of KV cache plus an assumed 1 to 2 GB of overhead. The formulas are on [[llm-quantization-explained]] and the rules are in [[methodology]]; the Qwen2.5-32B row is already in the [[vram-requirements]] table, so this page adds the per-card verdicts. Speed in tokens per second is unknown here, because no vendor page or project README read for this page names a machine and a figure.

## How much VRAM does a 32B model need?

Add three pieces, all in decimal GB (bytes / 1e9):

- Weights: `weights_GB = params x bits / 8 / 1e9`.
- KV cache: `2 x layers x kv_heads x head_dim x context x bytes_per_value`.
- Overhead: 1 to 2 GB, an assumption, not a measurement.

Bits per weight come from the [llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md), which prints them for Llama 3.1 8B; applying them to Qwen3-32B is an estimate, as on [[vram-requirements]]. Parameters, layers and heads come from the [Qwen3-32B model card](https://huggingface.co/Qwen/Qwen3-32B) (32.8B total, 64 layers, 64 query and 8 KV heads) and its [config.json](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json) (head_dim 128, bfloat16, so 2 bytes per value).

## How much VRAM for Qwen 32B at each quantization?

Qwen3 32B VRAM, weights only, from 32.8e9 parameters:

| Quant | Bits per weight | Weights | Weights + KV at 4,096 | Plus 1 to 2 GB overhead |
| --- | ---: | ---: | ---: | ---: |
| Q2_K | 3.1593 | 12.9531 GB | 14.0268 GB | 15.03 to 16.03 GB |
| Q3_K_M | 3.9960 | 16.3836 GB | 17.4573 GB | 18.46 to 19.46 GB |
| Q4_K_M | 4.8944 | 20.0670 GB | 21.1407 GB | 22.14 to 23.14 GB |
| Q5_K_M | 5.7036 | 23.3848 GB | 24.4585 GB | 25.46 to 26.46 GB |
| Q6_K | 6.5633 | 26.9095 GB | 27.9832 GB | 28.98 to 29.98 GB |
| Q8_0 | 8.5008 | 34.8533 GB | 35.9270 GB | 36.93 to 37.93 GB |

KV at 4,096 tokens: 2 x 64 x 8 x 128 x 4,096 x 2 = 1.0737 GB. It is a straight line in context: 2.1475 GB at 8,192 and 8.5899 GB at 32,768, the card's native length. The Qwen2.5-32B card gives the same layer and KV-head counts in its [config.json](https://huggingface.co/Qwen/Qwen2.5-32B/raw/main/config.json) (64 layers, 8 KV heads), so its KV figures match; its weights are 19.8835 GB at Q4_K_M, as in [[vram-requirements]]. How much quality each quant loses is unknown here; no source read for this page measures it.

## Can a 32B model run on 16GB of VRAM?

Not fully on the GPU at Q4_K_M. The [RTX 4060 Ti page](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4060-4060ti/) lists 16 GB GDDR6 as a configuration, and Q4_K_M weights alone are 20.067 GB, 4.067 GB over. Q3_K_M weights (16.3836 GB) also exceed 16 GB. Only Q2_K gets close: 15.03 to 16.03 GB at 4,096 context, which is borderline and leaves no room for longer context. The alternative is to keep some layers in system RAM, which the runtime must support, and speed with offload is unknown here. See [[16gb-vram-llm]] for the 16 GB class and [[run-llm-on-cpu]] for offload.

## Can a 32B model run on 24GB at Q4_K_M?

Yes, with a short context and a tight margin. NVIDIA lists 24 GB GDDR6X on the [RTX 4090](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/) and the [RTX 3090](https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/). Qwen3-32B at Q4_K_M and 4,096 context is 22.14 to 23.14 GB. At 8,192 context it is 20.067 + 2.1475 + 1 to 2 = 23.21 to 24.21 GB, so the top of that range is over 24 decimal GB. NVIDIA's pages say "24 GB" and the unit is not stated, so treat 24 as the ceiling. Q3_K_M at 4,096 (18.46 to 19.46 GB) leaves room for more context. Details are on [[24gb-vram-llm]].

## Does a 32 GB card change the answer?

Yes. The [RTX 5090 page](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/) lists 32 GB GDDR7. Q5_K_M at 4,096 is 25.46 to 26.46 GB and Q6_K is 28.98 to 29.98 GB. Q8_0 weights alone (34.8533 GB) do not fit. At the native 32,768 context the KV cache is 8.5899 GB, so Q4_K_M is 20.067 + 8.5899 + 1 to 2 = 29.66 to 30.66 GB. See [[32gb-vram-llm]].

## What about Gemma 4 31B?

Its weights are smaller but its KV cache is not. The [Gemma 4 31B card](https://huggingface.co/google/gemma-4-31B-it) lists 30.7B parameters, 60 layers and a 1,024-token sliding window; its [config.json](https://huggingface.co/google/gemma-4-31B-it/raw/main/config.json) shows 50 sliding layers with 16 KV heads of head dimension 256 and 10 global layers with 4 KV heads of head dimension 512. Q4_K_M weights are 30.7e9 x 4.8944 / 8 / 1e9 = 18.7823 GB.

Applying the KV formula to every layer with no window cap and 2 bytes per value gives an upper bound: 50 layers = 3.3554 GB and 10 layers = 0.3355 GB, 3.691 GB at 4,096 context. Total: 18.7823 + 3.691 + 1 to 2 = 23.47 to 24.47 GB. Capping the sliding layers at the 1,024-token window instead gives 1.1744 GB at 4,096 tokens (as on [[gemma-vram-requirements]]), or 18.7823 + 1.1744 + 1 to 2 = 20.96 to 21.96 GB. Ollama's Q4_K_M file for this model is 20 GB, above the formula's 18.7823 GB, so size from the file when you pull it. Whether a runtime stores less for sliding layers, or stores unified keys and values once, is unknown here; the card mentions unified keys and values on global layers without a byte count. Whether the 30.7B includes the roughly 550M vision encoder is also not stated. More model-by-model arithmetic is planned at [[gemma-vram-requirements]] and [[qwen-vram-requirements]].

## What if I need more than 32B?

A 70B-class model at Q4_K_M is over 43 GB of weights alone (the [llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md) lists Llama 3.1 70B at 43.1 GB); see [[vram-for-70b-model]]. For your own numbers, [[vram-calculator]] is planned.

## Questions

### How much VRAM for a 32B model?

About 22 to 23 GB at Q4_K_M with 4,096 context. For Qwen3-32B that is 20.067 GB of weights, 1.0737 GB of KV cache and an assumed 1 to 2 GB of overhead. Q8_0 needs 36.93 to 37.93 GB by the same arithmetic.

### Can a 32B model run on 16GB of VRAM?

Not fully on the GPU at Q4_K_M, since the weights are 20.067 GB. Q2_K is borderline at 15.03 to 16.03 GB and its quality loss is unknown here. Otherwise some layers must be offloaded to system RAM, with speed unknown.

### Can a 32B model run on 24GB at Q4_K_M?

Yes at a short context. 22.14 to 23.14 GB fits under 24 GB at 4,096 tokens, but 8,192 tokens reaches 23.21 to 24.21 GB. Lower the context or use Q3_K_M if it fails.

### How much VRAM for Qwen 32B?

Qwen3-32B has 32.8B parameters, so Q4_K_M weights are 20.067 GB and the KV cache is 1.0737 GB at 4,096 tokens. Qwen2.5-32B (32.5B) is 19.8835 GB of weights with the same KV figure.

### What is Qwen3 32B VRAM at its full context?

At 32,768 tokens the KV cache alone is 8.5899 GB, so Q4_K_M needs 29.66 to 30.66 GB, which fits a 32 GB card but not 24 GB.

## Sources read on 2026-10-13

- [llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md): bits per weight for Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0 (printed for Llama 3.1 8B); Llama 3.1 70B Q4_K_M 43.1 GB.
- [Qwen3-32B card](https://huggingface.co/Qwen/Qwen3-32B) and [config.json](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json): 32.8B parameters, 64 layers, 8 KV heads, head_dim 128, bfloat16 from config.json; 32,768 native context from the card (config.json max_position_embeddings reads 40960, not used).
- [Qwen2.5-32B config.json](https://huggingface.co/Qwen/Qwen2.5-32B/raw/main/config.json): 64 layers, 8 KV heads, bfloat16.
- [Gemma 4 31B card](https://huggingface.co/google/gemma-4-31B-it) and [config.json](https://huggingface.co/google/gemma-4-31B-it/raw/main/config.json): 30.7B parameters, 60 layers, sliding window 1024, roughly 550M vision encoder, "global layers feature unified Keys and Values"; config.json: 50 sliding and 10 full layers, attention_k_eq_v true.
- NVIDIA: [RTX 4060 Ti](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4060-4060ti/) 16 GB GDDR6 option, [RTX 4090](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/) and [RTX 3090](https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/) 24 GB GDDR6X, [RTX 5090](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/) 32 GB GDDR7.

## Related

- [[vram-requirements]] has the 7B to 70B table and formulas.
- [[vram-for-70b-model]] covers the next size up.
- [[16gb-vram-llm]], [[24gb-vram-llm]], [[32gb-vram-llm]] cover the card classes.
- [[qwen-vram-requirements]] and [[gemma-vram-requirements]] cover the model families.
- [[vram-calculator]], [[llm-quantization-explained]], [[run-llm-on-cpu]], [[methodology]].