Overview
gpt-oss-120b needs about 65.25 GB for its weights alone, so it does not fit in any single consumer GPU’s VRAM and only fits on a card of 80 GB or more; gpt-oss-20b needs about 13.76 GB of weights, so it fits a 16 GB card but not a 12 GB or 8 GB one without offload. Both numbers are the summed .safetensors sizes in the Hugging Face repos, not an estimate, and KV cache and overhead come on top. The method is the one on methodology and the general version is vram-requirements.
How much VRAM does gpt-oss-120b need?
At least 65.25 GB for the weights, plus KV cache, plus overhead. The gpt-oss-120b card lists 117B parameters with 5.1B active, says the model fits “into a single 80GB GPU (like NVIDIA H100 or AMD MI300X)”, and says it was post-trained with MXFP4 quantization of the MoE weights. The repo’s file listing shows 15 safetensors files totalling 65,248,893,184 bytes, which is 65,248,893,184 / 1e9 = 65.2489 decimal GB (60.77 GiB).
KV cache follows 2 x layers x kv_heads x head_dim x context x bytes_per_value. The config.json gives 36 layers, 8 key-value heads and head dimension 64. I assume 2 bytes per value (a 16-bit cache; I did not read the cache type your runtime will pick). That is 2 x 36 x 8 x 64 x 2 = 73,728 bytes per token. The config alternates sliding-window attention (window 128) and full attention, and how much cache a runtime actually allocates for the sliding layers is unknown here, so this is an upper bound that treats every layer as full attention.
| Context | KV cache, upper bound | Weights + KV + 1 to 2 GB assumed overhead |
|---|---|---|
| 4,096 | 0.3020 GB | 66.5509 to 67.5509 GB |
| 131,072 (config maximum) | 9.6637 GB | 75.9126 to 76.9126 GB |
The 1 to 2 GB overhead is an assumption, not a measurement. Even the long-context row is under 80 GB, which is consistent with the card’s 80 GB statement, but the margin is under 4 GB.
How much VRAM does gpt-oss-20b need?
About 13.76 GB for weights, so roughly 15 to 16 GB with a short context. The gpt-oss-20b card lists 21B parameters with 3.6B active and says the model can “run within 16GB of memory”. The repo files total 13,761,316,904 bytes in the top-level shards (the original/model.safetensors copy is 13,761,300,984 bytes, as on 8gb-vram-llm), which is 13.7613 GB either way (12.82 GiB). The config.json gives 24 layers, 8 KV heads and head dimension 64, so KV is 2 x 24 x 8 x 64 x 2 = 49,152 bytes per token under the same 2-byte assumption: 0.2013 GB at 4,096 tokens and 6.4425 GB at 131,072.
Weights + KV + overhead at 4,096 tokens is 13.7613 + 0.2013 + 1 to 2 = 14.9626 to 15.9626 GB. That arithmetic lands just under 16 GB and matches the card’s wording. At the full 131,072 context it is 21.2038 to 22.2038 GB, which no longer fits 16 GB.
What does “within 16GB of memory” mean?
It is the card’s own statement that the 20B model can run in 16 GB of memory; it does not say whether that is VRAM, system RAM or unified memory, and it does not name a context length. The arithmetic above shows why it is tight: weights are 13.76 GB, leaving about 2.2 GB for cache and runtime at a 16 GB budget. Treat it as a short-context figure.
Can gpt-oss-120b run on 24GB of VRAM?
Only by keeping most of the experts in system RAM. The weights are 65.2489 GB, so at least 65.2489 - 24 = 41.2489 GB cannot live on a 24 GB card even before cache and overhead. In practice the GPU also needs room for KV and compute buffers, so more spills over. Expert offload puts the expert weights in RAM and keeps the always-used layers on the GPU; the mechanism and flags are on moe-cpu-offload. The system therefore needs roughly 41 GB or more of free RAM on top of the OS (this is arithmetic from the file size, not a tested requirement). Speed in tokens per second is unknown: no vendor page or project README read for this page publishes a figure for this model on a named machine. See 24gb-vram-llm and ram-vs-vram.
Which GPU runs which gpt-oss model?
VRAM figures are from NVIDIA’s pages; the shortfall column is weights only (file size minus VRAM), before cache and overhead.
| GPU (VRAM, source) | gpt-oss-20b (13.7613 GB) | gpt-oss-120b (65.2489 GB) |
|---|---|---|
| RTX 3060 (12 GB GDDR6; page also lists an 8 GB variant) | Over by 1.7613 GB on 12 GB, 5.7613 GB on 8 GB: needs some offload | Over by 53.2489 GB: expert offload and large RAM |
| RTX 3090 (24 GB GDDR6X) | Fits: 24 - 13.7613 = 10.2387 GB left for cache and overhead | Over by 41.2489 GB: expert offload |
| RTX 5090 (32 GB GDDR7) | Fits: 18.2387 GB left | Over by 33.2489 GB: expert offload |
| RTX PRO 6000 (96 GB GDDR7 ECC) | Fits | Fits: 96 - 65.2489 = 30.7511 GB left |
Per card: on a 3060 12 GB the 20B model does not fit whole (see 12gb-vram-llm); on 8 GB the gap is larger still (8gb-vram-llm). A 3090 and a 5090 run the 20B in VRAM and the 120B only with offload (rtx-3090-llm, rtx-5090-llm). The RTX PRO 6000 is the only card in this table that holds the 120B whole, with the 96 GB the NVIDIA page lists. A 48 GB total from two cards is still under 65.25 GB (48gb-vram-llm).
Why is the active parameter count not the VRAM number?
Memory capacity follows total parameters; per-token reads follow active parameters. gpt-oss-120b stores 117B parameters and uses 5.1B per token, an active share of 5.1 / 117 = 4.36%; gpt-oss-20b is 3.6 / 21 = 17.14%. All of it still has to be stored somewhere, which is why the file sizes above decide fit, and why offloading experts to RAM is possible at all.
Questions
How much VRAM does gpt-oss-120b need?
About 65.25 GB for weights (the repo’s safetensors total 65,248,893,184 bytes), plus 0.30 GB of KV cache at 4,096 tokens under an upper-bound 2-byte assumption, plus an assumed 1 to 2 GB overhead, so roughly 66.6 to 67.6 GB. The model card says it fits a single 80GB GPU.
Can gpt-oss-120b run on a single 24GB GPU?
Not entirely in VRAM: the weights exceed 24 GB by at least 41.25 GB. It can be run with expert layers offloaded to system RAM, which needs roughly 41 GB or more of RAM free by this arithmetic. Speed is unknown; no sourced figure exists on this page. See moe-cpu-offload.
Can gpt-oss-20b run on 8GB VRAM or an RTX 3060 12GB?
Not whole. The 20B weights are 13.7613 GB, which is 1.7613 GB over a 12 GB card and 5.7613 GB over 8 GB. It needs partial offload to system RAM on those cards; the resulting speed is unknown here.
What does “within 16GB of memory” mean on the gpt-oss-20b card?
The card says the model can run within 16GB of memory without naming the memory type or a context length. Our arithmetic gives 14.9626 to 15.9626 GB at 4,096 tokens, so the claim holds for short contexts and not for the full 131,072 maximum.
Is the 120B one of the 117B or 120B parameters?
The card gives 117B total parameters with 5.1B active. The name “120b” is rounded; this page uses the card’s 117B and the repo’s file size.
Sources read on 2026-10-13
- gpt-oss-20b model card: 21B parameters, 3.6B active, “within 16GB of memory”, MXFP4 post-training of the MoE weights.
- gpt-oss-120b model card: 117B parameters, 5.1B active, single 80GB GPU statement, MXFP4.
- gpt-oss-20b files and gpt-oss-120b files: safetensors byte sizes (13,761,316,904 and 65,248,893,184 summed).
- gpt-oss-20b config.json and gpt-oss-120b config.json: layers (24, 36), 8 KV heads, head dim 64, sliding window 128, 131,072 max positions, MXFP4.
- NVIDIA RTX PRO 6000: 96 GB GDDR7 with ECC (Workstation Edition).
- NVIDIA RTX 5090: 32 GB GDDR7.
- NVIDIA RTX 3090 family: 24 GB GDDR6X.
- NVIDIA RTX 3060 family: 12 GB / 8 GB GDDR6.
- llama.cpp quantize README: read for the formula’s bits-per-weight context; it does not mention MXFP4, so no llama.cpp quantization figure is used for gpt-oss.
Related
- moe-cpu-offload covers the expert offload mechanism.
- 24gb-vram-llm, 12gb-vram-llm, 8gb-vram-llm, 48gb-vram-llm size by card class.
- ram-vs-vram explains the memory difference.
- vram-requirements and vram-calculator are the general formula and tool.
- rtx-5090-llm and rtx-3090-llm cover the two consumer cards.
- methodology is the citation rule.