Best LLM for 48GB VRAM: Cards and Models That Fit

Overview

The best LLM for 48GB VRAM is a 70B-class dense model at roughly 4-bit quantization with a short context: Qwen2.5-72B at Q4_K_M is 44.4779 GB of weights plus 1.3422 GB of KV cache at 4,096 tokens, which lands at 46.8 to 47.8 GB, so it fits only barely. Long context does not fit. The formulas come from methodology and llm-quantization-explained. Every GPU figure is from a vendor page read on 2026-10-13, and a figure the page does not give is written as unknown. No tokens-per-second figure appears because none was read from a vendor page naming the machine.

Which cards have 48 GB of VRAM?

Two NVIDIA workstation cards and the AMD Radeon PRO W7900 list 48 GB on their own pages.

The A6000 page also lists a 2-way NVLink bridge, which it says connects 2 RTX A6000 cards. The “112 gigabytes per second” it quotes is the bridge’s link bandwidth, not the card’s memory bandwidth.

What is the cheapest way to get 48GB of VRAM?

Unknown. This page states no prices because vendor pages do not list street prices and resale prices change; check the current retailer price. What can be said without a price is the shape of the options:

• One 48 GB card (the table above). One memory pool, so the whole model sits on one device.
• Two 24 GB cards. Totals 48 GB but is not one pool. llama.cpp-style runtimes split the model by layers, so each card holds its share of the weights and its share of the KV cache, and each card needs its own overhead. See multi-gpu-llm and dual-rtx-3090-llm for the build side and 24gb-vram-llm for what a single 24 GB card holds.
• Used datacenter cards. Covered on used-datacenter-gpus.
• Modded RTX 4090 48GB (status only). No NVIDIA product page that was read lists a 48 GB RTX 4090, so there is no vendor specification to cite. Its capacity, support and warranty are unknown here.

No price range is stated or matched to a part on this page.

What can you run with 48GB VRAM?

The sizing rule from methodology: weights_GB = params × bits ÷ 8 ÷ 1e9, plus KV cache = 2 × layers × kv_heads × head_dim × context × bytes_per_value, plus an assumed 1 to 2 GB overhead (an assumption, not a measurement). Inputs for Qwen2.5-72B: 72.7B parameters on its model card, and in its config.json 80 layers, 8 KV heads, hidden size 8192 over 64 heads (head dimension 128) and bfloat16 (2 bytes). Bits per weight are 4.8944 for Q4_K_M and 8.5008 for Q8_0 from the llama.cpp quantize README. The README’s own Llama 3.1 70B Q4_K_M file is 43.1 GB.

• Q4_K_M weights: 72.7e9 × 4.8944 ÷ 8 ÷ 1e9 = 44.4779 GB.
• Q8_0 weights: 72.7e9 × 8.5008 ÷ 8 ÷ 1e9 = 77.2510 GB.
• KV per token: 2 × 80 × 8 × 128 × 2 = 327,680 bytes.

So 48 GB is the first tier where a 70B-class model at Q4 loads fully on GPU, with little room left. Context past a few thousand tokens pushes it over. Smaller models such as a 32B leave room for much longer context: the same arithmetic is on vram-for-32b-model and vram-for-70b-model. Whether 48 GB is better than 32 GB depends on that choice; see 32gb-vram-llm.

Is GPT-OSS-120B the best LLM that fits in 96GB VRAM?

It is a strong fit, but “best” is a judgment this page does not make. The gpt-oss-120b model card lists 117B parameters with 5.1B active, MoE weights post-trained with MXFP4 quantization, and says it runs “on a single 80GB GPU”. The repo’s safetensors files total 65,248,893,184 bytes, which is 65.2489 decimal GB (worked on gpt-oss-vram-requirements), so the weights fit 96 GB with 30.7511 GB left for KV cache and overhead. The 80 GB statement is the card’s, and it is below 96 GB.

KV cache from its config.json: 36 layers (18 sliding-window at 128 tokens, 18 full attention), 8 KV heads, head dimension 64, maximum 131,072 positions. KV bytes per value are not given there; assuming 2, treating all 36 layers as full attention (an upper bound): 2 × 36 × 8 × 64 × 131,072 × 2 = 9.6637 GB at the full 131,072 tokens. Whether a runtime stores fewer bytes for the sliding layers is unknown here.

What do you run at 96GB and 128GB?

The NVIDIA RTX PRO 6000 Blackwell Workstation Edition page lists “96 GB GDDR7 with error-correcting code (ECC)”, “1792 GB/sec” memory bandwidth and 600 W max power. That is the one 96 GB part read for this page. A 128 GB figure comes from combining cards or a unified-memory system, not from a part read here, so its bandwidth and layout are unknown.

Qwen2.5-72B using the same formulas, and its card’s 131,072-token maximum (KV 327,680 × 131,072 = 42.9497 GB):

Questions

48GB VRAM users, what are your daily drivers?

This page cannot report community usage, only what fits. On 48 GB the sourced answer is a 70B-class dense model at about 4-bit with short context, or a smaller model with long context. Qwen2.5-72B at Q4_K_M and 4,096 tokens needs 46.8201 to 47.8201 GB by the formula above, so it is tight.

What is the cheapest way to get 48GB of VRAM?

Unknown; prices are not stated here, so check the current retailer price. The options are one 48 GB card (RTX 6000 Ada or RTX A6000, both listed by NVIDIA at 48 GB GDDR6), two 24 GB cards split by layers, or used datacenter parts. AMD’s Radeon PRO W7900 page also lists 48 GB GDDR6.

Is GPT-OSS-120B the best LLM that fits in 96GB VRAM?

It fits, since its model card says it runs on a single 80GB GPU, but this page does not rank models. Its weights are 65.2489 GB (repo file total), and its KV cache is at most 9.6637 GB at 131,072 tokens under the stated assumptions.

What do I run at 128GB VRAM?

Per the formulas above, Qwen2.5-72B at Q8_0 with its full 131,072-token context needs 121.2007 to 122.2007 GB, which fits in 128 GB with little margin. No 128 GB part was read, so its memory bandwidth is unknown.

Can two 24GB cards act as 48GB VRAM?

Only for capacity, as a layer split: each card holds part of the model and its KV cache, plus its own overhead, so a model needing 47 GB on one card may not fit across two. Speed is unknown here. See multi-gpu-llm.

Is there a 48GB RTX 4090?

No NVIDIA page read for this site lists one, so the status is unknown and no specification can be cited.

Sources read on 2026-10-13

• NVIDIA RTX 6000 Ada: 48GB GDDR6 with ECC, 300 W; bandwidth and interface not stated.
• NVIDIA RTX A6000: 48 GB GDDR6 with ECC, 300 W, 2-way NVLink bridge at up to 112 GB/s link bandwidth.
• NVIDIA RTX PRO 6000 Blackwell: 96 GB GDDR7 with ECC, 1792 GB/sec, 600 W.
• AMD Radeon PRO W7900: 48 GB GDDR6, 384-bit, 864 GB/s peak, 295 W TBP, 750 W minimum PSU; read 2026-10-13 after two HTTP 503 responses.
• gpt-oss-120b card and config.json: 117B parameters, 5.1B active, MXFP4, single 80GB GPU; 36 layers, 8 KV heads, head dimension 64, sliding window 128, 131,072 positions.
• Qwen2.5-72B card and config.json: 72.7B parameters; 80 layers, 8 KV heads, 64 heads, hidden size 8192, bfloat16, 131,072 positions.
• llama.cpp quantize README: Q4_K_M 4.8944 and Q8_0 8.5008 bits per weight; Llama 3.1 70B Q4_K_M 43.1 GB.

Related

• 32gb-vram-llm and 24gb-vram-llm cover the tiers below.
• multi-gpu-llm and dual-rtx-3090-llm cover splitting across cards.
• gpt-oss-vram-requirements and vram-for-70b-model go deeper by model.
• used-datacenter-gpus covers used parts.
• methodology and llm-quantization-explained define the formulas.
• vram-requirements is the general table.