LLM Hardware Guide

RAM vs VRAM for LLM: What Each One Does

By LLM Hardware Guide Editors8 min read

View markdown source

AI agents: read this first: llmhardwareguide.com/start-here

This is LLM Hardware Guide (llmhardwareguide.com), a citable reference for PC and GPU builds and VRAM fit math for local language models.

  1. Route, do not crawl. Fetch llmhardwareguide.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Do not invent a number. If a spec is missing, keep it unknown. Fit claims use the formula on the quantization page. Do not add tokens-per-second figures.
  4. Cite. Link the page by title and URL, e.g. VRAM requirements, with last_updated. Mac questions belong on llmformac.com.

Overview

RAM vs VRAM for an LLM comes down to this: VRAM is the memory on the graphics card that holds the layers the GPU computes, and system RAM is the memory on the CPU side that holds whatever does not fit, so a model that fits entirely in VRAM stays on the GPU and a model that does not either spills into RAM or fails. System RAM does not become VRAM; a runtime places some layers on each. The sizing formulas are on llm-quantization-explained and the method is in methodology. If you are on a Mac with unified memory, this split does not apply: see how much RAM for a local LLM.

What is the difference between RAM and VRAM for LLMs?

Both hold the same bytes: weights, KV cache and overhead. What differs is which processor sits next to the memory and how wide the path is. VRAM is soldered to the GPU. NVIDIA lists the RTX 5090 with 32 GB GDDR7 on a 512-bit interface and the RTX 4090 with 24 GB GDDR6X on a 384-bit interface. Neither page states a memory bandwidth in GB/s, so the GPU bandwidth is unknown here.

System RAM sits on the CPU’s memory bus. Intel’s Core Ultra 9 285K page lists up to DDR5 6400 MT/s, 2 memory channels and 256 GB maximum. A theoretical peak is transfers per second x bytes per transfer x channels. With the page’s 6400 MT/s and 2 channels, and assuming the standard 8 bytes (64 bits) per channel, which the page does not state: 6400e6 x 8 x 2 / 1e9 = 102.4 GB/s. That is a ceiling, not a measured rate, and Intel’s page lists no bandwidth figure to compare with it. No tokens-per-second figure is published here.

VRAMSystem RAM
Attached toThe GPUThe CPU
Fixed byThe card you buyYour DIMMs and motherboard
Example from vendor pages32 GB, 512-bit (RTX 5090)Up to 256 GB, 2 channels, DDR5 6400 MT/s (Core Ultra 9 285K)
BandwidthUnknown (not stated on the NVIDIA pages)102.4 GB/s theoretical peak, from the formula above

Can you run an LLM on RAM instead of VRAM?

Yes. A model loaded entirely in system RAM runs on the CPU. Ollama’s FAQ says the Processor column of ollama ps shows where the model landed: 100% CPU means entirely in system memory, and a split such as 48%/52% CPU/GPU means partly on the GPU and partly in system memory. The full CPU-only picture is on run-llm-on-cpu.

Can I use system RAM as VRAM?

Not as a setting that makes RAM behave like VRAM. What you can do is partial offload. llama.cpp’s CLI README defines -ngl / --n-gpu-layers as the “max. number of layers to store in VRAM”, with a default of auto. Layers not stored in VRAM stay on the CPU side. The same README lists --cpu-moe (“keep all Mixture of Experts (MoE) weights in the CPU”) and --n-cpu-moe N for MoE models; see moe-cpu-offload. It also lists --fit (“whether to adjust unset arguments to fit in device memory”, default on) with a --fit-target default of 1024 MiB.

How much system RAM do I need for GPU offload?

At least the bytes that do not fit in VRAM, plus the operating system and other programs. A worked split, using the Qwen2.5-72B numbers from vram-requirements: weights at Q4_K_M are 44.4779 GB (72.7e9 parameters x 4.8944 bits / 8 / 1e9, bits per weight from the llama.cpp quantize README), and KV cache at 4,096 tokens is 1.3422 GB. On a 24 GB card with this site’s assumed 1 to 2 GB overhead (worked at the 2 GB end; the 1 GB end is given after step 4):

  1. VRAM left for weights: 24 - 2 - 1.3422 = 20.6578 GB.
  2. Weight per layer, assuming 80 equal layers and ignoring the embedding and output tensors: 44.4779 / 80 = 0.5560 GB.
  3. Layers on the GPU: 20.6578 / 0.5560 = 37.16, so 37 layers.
  4. Layers left for RAM: 80 - 37 = 43 layers, 43 x 0.5560 = 23.9069 GB.

At 1 GB overhead the same steps give 24 - 1 - 1.3422 = 21.6578 GB for weights, 38 layers on the GPU and 42 layers (23.3510 GB) left for RAM. So that setup needs about 23.4 to 23.9 GB of free system RAM for the spilled layers, before the operating system. The uniform-layer assumption makes this an estimate; the real split depends on the runtime. Whether it is fast enough is unknown: no vendor or project page names a machine and a rate for it.

Why does a full VRAM make things slower?

Each generated token reads the weights, and the layers in RAM are read over the CPU’s memory path instead of the GPU’s. This page can state the direction, not the size: no cited source gives a measured rate for a split model, and the GPU bandwidth is unknown above. Ollama’s FAQ says that when a model fits entirely on one GPU it loads there, which “typically provides the best performance as it reduces the amount of data transferring across the PCI bus during inference”. If the load is not reaching the GPU at all, see ollama-not-using-gpu; for failures when allocation exceeds VRAM, see cuda-out-of-memory-llm.

How do I increase VRAM for an LLM?

On a discrete card you cannot: the VRAM is fixed by the card, so more VRAM means a different card or a second one. Check what you have with how-to-check-vram and size the target with vram-requirements. Context length is the setting that does change memory use, because KV cache grows in a straight line with it. On integrated AMD graphics under Linux, the kernel’s amdgpu module parameters list gttsize for restricting the GTT domain, default “-1 (Use value specified by TTM)”; the page marks it deprecated and says it is for testing, so this page does not recommend it as a way to add memory.

Does Gemma 4 need RAM or VRAM?

Both, in the same way as any other model: the file plus KV cache plus overhead has to land somewhere. This page has not read a Gemma 4 model card, so no Gemma 4 size is stated here. See gemma-vram-requirements for the sizes and fit by card, and vram-calculator for the formula.

Questions

What is the difference between RAM and VRAM for LLMs?

VRAM is memory attached to the GPU and RAM is memory attached to the CPU. Both can hold weights and KV cache, but the GPU computes layers stored in VRAM and the CPU computes layers left in RAM. Vendor pages give bus widths (512-bit on the RTX 5090, 384-bit on the RTX 4090) but no GPU bandwidth figure, which is unknown here.

Can you run an LLM on RAM instead of VRAM?

Yes. A model loaded fully in system memory runs on the CPU, and Ollama’s ollama ps reports it as 100% CPU. It needs enough free RAM for the weights, KV cache and overhead. See run-llm-on-cpu.

Can I use system RAM as VRAM?

Not as a drop-in. You can offload: llama.cpp’s -ngl sets how many layers are stored in VRAM, and the remaining layers stay on the CPU side in RAM. The CPU computes those layers, so RAM is a spill area, not extra VRAM.

How much system RAM do I need, separate from VRAM?

Enough for the layers that do not fit in VRAM plus the operating system. In the worked example, a 72B model at Q4_K_M on a 24 GB card leaves 42 to 43 of 80 layers, about 23.4 to 23.9 GB, in RAM (1 to 2 GB overhead assumed). That figure is an estimate that assumes equal layers.

How do I increase VRAM for an LLM?

You cannot raise it on a discrete GPU; you buy a card with more, or add a second. Lowering context length or the quantization level reduces what you need. The amdgpu gttsize parameter for integrated graphics is marked deprecated and for testing in the kernel documentation.

Sources read on 2026-10-13

Backlinks