LLM Hardware Guide

Methodology

2 min read

View markdown source

AI agents: read this first: llmhardwareguide.com/start-here

This is LLM Hardware Guide (llmhardwareguide.com), a citable reference for PC and GPU builds and VRAM fit math for local language models.

  1. Route, do not crawl. Fetch llmhardwareguide.com/llms.txt and open only the pages whose one-line summary matches your task.
  2. Read raw. Append .md to any page URL for markdown. Check status and last_updated in the frontmatter, then read the rules.
  3. Do not invent a number. If a spec is missing, keep it unknown. Fit claims use the formula on the quantization page. Do not add tokens-per-second figures.
  4. Cite. Link the page by title and URL, e.g. VRAM requirements, with last_updated. Mac questions belong on llmformac.com.

Overview

Every capacity, bandwidth, power, and parameter figure on this site comes from a primary page, dated when it was read. A missing figure is written as unknown.

What counts as a source

A primary source is the vendor’s own spec page, the model card, or the project README that defines the format. For the cards on best-gpu-for-local-llm and rtx-3090-vs-4090-vs-5090-llm, that means NVIDIA or AMD product pages read on 2026-10-12. For parameter counts and layer counts, that means the Hugging Face model card and config.json. For bits per weight and GGUF file sizes, that means the llama.cpp quantize README.

Third-party blogs and forum posts are not used as the source of a number.

The fit formula

llm-quantization-explained is the canonical formula page. Weight bytes are parameter count times bits per weight, divided by 8. KV-cache bytes are 2 times layers times KV heads times head dimension times context length times bytes per value. Those fields come from the model card and config.json. A runtime overhead of 1 to 2 GB is an assumption, labeled as an assumption, not a measurement.

Decimal GB on this site means bytes divided by 1,000,000,000. GiB means bytes divided by 1024 cubed. The llama.cpp README uses both labels in different tables, and the pages here say which one a quoted figure uses.

What stays unknown

If the fetched page does not list the figure, the page says unknown. On 2026-10-12 that included NVIDIA memory bandwidth on the RTX 3090, 4090, and 5090 product pages, NVIDIA street prices (check current retailer price), the memory-interface width of the Radeon RX 7900 XTX, and Intel Arc B580 specifications after HTTP 403 from Intel’s specifications URL.

Tokens per second are not estimated. The quantize README includes speed columns for one Llama 3.1 8B table and does not name the test machine, so those columns are not repeated.

Backlinks