Best Budget GPU for Local LLMs: VRAM per Dollar (2026)

Overview

The best budget GPU for local AI is the cheapest card that has enough confirmed VRAM for the model you want, which on the vendor pages read today means a 16 GB card if you buy new and a 24 GB card if you buy used. Rank by gigabytes, not by brand: a model either fits in VRAM or it spills to system RAM. This page does not state any price, because none of the vendor pages read lists one (NVIDIA’s own page shows the placeholder “Starting at $XXX.XX.”); check current retailer price, then divide it by the GB column below to get your own price per GB. Fit math follows methodology. Tokens per second: unknown, because none of these vendor pages publishes one.

Which budget GPU has the most VRAM per dollar?

Unknown, because it needs prices and this site does not quote street prices. The part we can verify is the numerator. Use this formula once you have a dated retailer price: price_per_GB = price / VRAM_GB. Here is the VRAM each vendor page confirms, read on 2026-10-13.

What is the best budget GPU for local AI if I buy new?

A 16 GB card: the RTX 5060 Ti 16 GB or the RX 9060 XT 16 GB. Both pages list 16 GB on a 128-bit bus, so they fit the same models by capacity; which is faster is unknown. Details are on rtx-5060-ti-16gb-llm. The trap is the RTX 5060 Ti, which NVIDIA lists as “16 GB / 8 GB GDDR7”, so confirm the memory size on the box. The RTX 5060 is 8 GB only, and 8 GB is covered on 8gb-vram-llm. The RX 9060 XT page shows a maximum of 16 GB and does not say whether a smaller version exists, so check that too. AMD runtime support is a separate question: amd-gpu-for-local-llm.

Is the Intel Arc B580 a budget option for local LLMs?

It has 12 GB, confirmed: Intel’s specification page lists “12 GB GDDR6”, 192-bit, 456 GB/s and 190 W TBP. That is more than 8 GB and less than 16 GB. Whether your runtime (llama.cpp, Ollama) uses the card well is not answered by that page, so it is unknown here; see intel-arc-for-local-llm. For the 12 GB class see 12gb-vram-llm.

What is the cheapest GPU with 24GB of VRAM?

Of the cards read here, the used RTX 3090 and the used Tesla P40 both list 24 GB. Which is cheaper depends on prices this page does not quote. They are different products. The RTX 3090 is a desktop card at 350 W with a 750 W required system power rating on NVIDIA’s page (rtx-3090-llm). The P40 is a 2016 data-center card: NVIDIA’s datasheet gives 346 GB/s, 250 W max, PCI Express 3.0 x16 and a dual-slot full-height form factor. The datasheet does not state the cooling method, so whether it needs a fan added is unknown; read the listing. More such cards: used-datacenter-gpus.

What does 16 GB versus 24 GB buy you?

Weights_GB = params x bits / 8 / 1e9, plus KV cache, plus an assumed 1 to 2 GB overhead, as defined on methodology. Bits per weight use 4.8944, the Q4_K_M figure in the quantize README, which the README lists for Llama-3.1-8B; applying it to the Qwen models is an approximation. Parameters, layers and KV heads are from the Qwen2.5 model cards (7B, 14B, 32B) and each config.json (head dimension 128, bfloat16, 2 bytes). KV bytes = 2 x layers x kv_heads x 128 x 4,096 x 2. Decimal GB.

The 14B row is under 12 GB only by 0.2 GB at the high overhead, so treat 12 GB as tight, not safe. The 32B row is the reason 24 GB matters: it is the first tier in this table that holds it, and with only 1.0428 to 2.0428 GB to spare, longer context will not fit. Long context grows the KV term in a straight line. See vram-requirements, 16gb-vram-llm and 24gb-vram-llm.

What are the risks of buying a used GPU for LLMs?

No primary source quantifies these, so none is quantified here. Things to check before you pay: the listing must show the card’s real memory size (NVIDIA’s own page lists the RTX 5060 Ti as “16 GB / 8 GB GDDR7”); a 350 W card needs a power supply to match NVIDIA’s 750 W system rating for the 3090; a data-center card such as the P40 may need its own cooling and may have no video output, neither of which the datasheet states, so both are unknown; and warranty and wear are unknown for any used card. A model that does not fit ends in cuda-out-of-memory-llm or slow offload (run-llm-on-cpu). Two cards are an option but scaling is covered on dual-rtx-3090-llm.

Is it a good time to buy?

Unknown. This page has no price history and no vendor page read states one. Buy against a model you can name: if it fits 16 GB, do not pay for 24 GB.

Questions

What is the best budget GPU for local AI?

The cheapest card with enough confirmed VRAM for your model. New, that is a 16 GB RTX 5060 Ti or RX 9060 XT (both list 16 GB on a 128-bit bus); used, a 24 GB RTX 3090 or Tesla P40. Check current retailer price.

What is the cheapest GPU with 16GB or 24GB of VRAM?

Price is unknown here. For 16 GB, the vendor pages confirm the RTX 5060 Ti (16 GB / 8 GB versions) and RX 9060 XT (max 16 GB). For 24 GB, the used RTX 3090 and Tesla P40 both list 24 GB.

What is the best GPU for LLM under $500orunder$300?

Unknown. No vendor page read lists a price, and this site does not quote street prices. Filter by VRAM first: 12 GB (Arc B580) or 16 GB cards, then check dated retailer prices against your budget.

How much VRAM do I need on a budget?

By the table above, 16 GB holds a 14B model at Q4_K_M with 4,096 context (10.7988 to 11.7988 GB). A 32B model needs 21.9572 to 22.9572 GB, so it needs 24 GB.

Is a used RTX 3090 worth it for local LLMs?

It has 24 GB GDDR6X per NVIDIA, which holds a 32B Q4_K_M model by the arithmetic above. Value depends on price and the used-card checks above, both unknown here. See rtx-3090-llm.

Is the Arc B580 good for local LLMs?

It has 12 GB GDDR6 at 456 GB/s per Intel. Software support and speed are unknown here. See intel-arc-for-local-llm.

Sources read on 2026-10-13

• NVIDIA RTX 5060 family: 5060 Ti “16 GB / 8 GB GDDR7”, 5060 8 GB GDDR7, 128-bit, 180 W and 145 W total graphics power (reference design; partner cards may vary); no bandwidth; price placeholder only.
• AMD Radeon RX 9060 XT: Max Memory Size 16 GB, GDDR6, 128-bit, up to 320 GB/s, typical board power 160W (fetched with curl after WebFetch returned HTTP 503).
• NVIDIA RTX 3090 family: 24 GB GDDR6X, 384-bit, 350 W, 750 W required system power; no price or bandwidth.
• Intel Arc B580 specifications: 12 GB GDDR6, 192-bit, 456 GB/s, 190 W TBP.
• NVIDIA Tesla P40 datasheet (PDF, Sep 2016): 24 GB, 346 GB/s, PCIe 3.0 x16, dual slot, 250 W max; no cooling statement.
• llama.cpp quantize README: Q4_K_M bits per weight 4.8944 (listed for Llama-3.1-8B).
• Qwen2.5 7B, 14B, 32B cards and config.json: parameter counts, layers (28, 48, 64), KV heads (4, 8, 8), head dimension 128, bfloat16.
• AMD Instinct MI50 page (https://www.amd.com/en/products/accelerators/instinct/mi50.html) timed out and later returned HTTP 404. Only the ROCm table’s 32 GiB is cited, as on used-datacenter-gpus.

Related

• rtx-5060-ti-16gb-llm covers the 16 GB new cards.
• used-datacenter-gpus and rtx-3090-llm cover used 24 GB cards.
• intel-arc-for-local-llm covers Arc.
• best-gpu-for-local-llm, 16gb-vram-llm, 24gb-vram-llm, 12gb-vram-llm and 8gb-vram-llm cover cards by tier.
• vram-requirements and methodology are the formula and the citation rule.