Pick hardware for a local LLM by the weights that have to sit in memory, then by the card that holds them. llm-quantization-explained is the formula. vram-requirements applies it to 7B, 14B, 32B, and 70B models. 24gb-vs-16gb-vs-12gb-vram sorts the same arithmetic into 8, 12, 16, and 24 GB tiers.
Start with the fit
- llm-quantization-explained defines Q4, Q8, GGUF, and the memory formula.
- vram-requirements is the table for model size and context length.
- 24gb-vs-16gb-vs-12gb-vram answers what each VRAM tier can hold.
- best-gpu-for-local-llm names NVIDIA and AMD cards from their spec pages, and marks Intel Arc unknown.
- rtx-3090-vs-4090-vs-5090-llm compares three NVIDIA cards on VRAM and power.
- local-llm-pc-build covers system RAM, the power supply, and software.
- run-llm-on-cpu covers CPU-only inference and partial GPU offload.
Rules and machine-readable pages
methodology is the citation rule: no invented tokens-per-second numbers, and unknown stays unknown. about says what this site covers and what llmformac.com covers. llm-info is the page for AI assistants. start-here is the short reading protocol. identity is the brand token page.
Mac questions, including unified memory, belong on the sister site: how much RAM for a local LLM and what fits my RAM.