Used Datacenter GPUs for Local LLMs: Tesla P40, MI50, V100, A100

Overview

Used datacenter GPUs such as the Tesla P40 (24 GB), Tesla V100 (16 or 32 GB) and Instinct MI50 (32 GiB) hold more model per card than most consumer GPUs, but the vendor pages publish memory, bandwidth and power only: price, tokens per second, cooling for your case and driver support for your software stack are separate checks, and the MI50 is marked unsupported in current ROCm. Every figure below is from a vendor page read on 2026-10-13, and anything those pages do not state is written as unknown. Fit math follows methodology. For the consumer alternative see rtx-3090-llm; for a wider ranking see budget-gpu-for-local-llm.

What do the vendor pages say about each card?

Notes on the table. The V100 datasheet gives 32GB / 16GB with no unit definition, and the P40 datasheet says only “24 GB”. Whether those are decimal GB or GiB is unknown, so the fit math below treats them as 24e9 and 32e9 bytes at worst. The MI50 row is AMD’s ROCm table, which lists 32 GiB (34.3597 decimal GB, 32 x 2^30 / 1e9); AMD’s own MI50 page returned HTTP 503 on three tries and HTTP 404 on a later direct request, so bandwidth, power and cooling are unknown here. The H100 page’s body text also mentions “188GB HBM3” for the NVL without explaining it; the spec table’s 94 GB per card is used. Tokens per second for all of these cards: unknown, no vendor page publishes one. Price: unknown, check current retailer price.

Is a Tesla P40 LLM build still worth it?

It is the cheapest-looking way to get 24 GB of memory on one card, and the datasheet confirms the 24 GB, but it is a 2016 inference card and whether it is worth it depends on a price this page does not state and speed this page cannot measure. The datasheet’s 346 GB/s is the only bandwidth figure NVIDIA gives for it. Against a used RTX 3090 (see rtx-3090-llm), the datasheet calls the P40 “Server-Optimized for Data Center Deployment” and does not state its cooling, so confirm cooling and display output in the listing. Whether current llama.cpp, Ollama or vLLM builds still target this architecture is unknown here: no software support page was read for it. Check the project’s own docs before buying (llama-cpp-vs-ollama-vs-vllm covers the three).

Is the MI50 32GB worth it for local LLMs?

Only if you accept running without official ROCm support. AMD’s ROCm system requirements list the Instinct MI50 (GCN5.1, gfx906) with an unsupported mark, defined as: “The current ROCm release does not support this hardware.” The same page says “The HIP runtime might continue to run applications for an unsupported GPU, but prebuilt ROCm libraries are not officially supported and will cause runtime errors.” So the 32 GiB is real on paper, and the software path is the risk. Whether a given llama.cpp or Ollama build works on gfx906 is unknown here. General AMD guidance is on amd-gpu-for-local-llm.

Is the Tesla V100 32GB worth it in 2026?

The datasheet confirms 32 GB (and a 16 GB version) of HBM2 at 900 GB/s, which is well above the P40’s 346 GB/s. Two cautions come from the datasheet itself. The thermal solution is listed as passive, so it relies on server-style airflow; how to cool it in a desktop case is unknown here. And the SXM2 version uses NVLink as its interface, not a PCIe slot, so it does not drop into a normal motherboard. Worth it in 2026 depends on price (unknown) and software support (unknown here).

What fits in these cards?

Use Qwen2.5-32B at Q4_K_M as the worked case. Inputs: 32.5B parameters, 64 layers, 8 KV heads and a head dimension of 5120 / 40 = 128, with torch_dtype bfloat16, from the model card and config.json; Q4_K_M is 4.8944 bits per weight and Q8_0 is 8.5008, from the llama.cpp quantize README, where they are measured on Llama-3.1-8B, so applying them to Qwen2.5-32B is an approximation. Fp16 KV cache (2 bytes per value) is assumed; see methodology.

• weights_GB = params x bits / 8 / 1e9 = 32.5e9 x 4.8944 / 8 / 1e9 = 19.8835 GB
• KV cache = 2 (K and V) x layers 64 x KV heads 8 x head dim 128 x context x 2 bytes (fp16 assumed): 1.0737 GB at 4,096 tokens, 8.5899 GB at 32,768
• overhead: an assumed 1 to 2 GB, not a measurement

At Q8_0 the weights alone are 32.5e9 x 8.5008 / 8 / 1e9 = 34.5345 GB, over the MI50’s 34.3597 GB. A 70B-class model is larger: the README lists Llama 3.1 70B at 43.1 GB for Q4_K_M, which exceeds any 24 or 32 GB card before KV cache, and the KV term for that model is unknown here (its config was not read). On an A100 80GB, 80 - 43.1 = 36.9 GB would remain for cache and overhead, assuming both figures are decimal GB. Two cards are the usual route past one card’s limit: see multi-gpu-llm, multi-gpu-pcie-and-psu and 48gb-vram-llm. A fit that fails ends in cuda-out-of-memory-llm. Method and the full formula: llm-quantization-explained, vram-requirements.

What goes wrong with used datacenter cards?

This site has not measured any of these cards. The vendor-checkable parts are these. Passive cooling: the V100 datasheet says passive. Variants: the V100 comes in 16 GB and 32 GB, and the MI50 in 16 GiB and 32 GiB per AMD’s ROCm table, so read the listing for the exact one. Form factor: V100 SXM2 is not a PCIe card. Power: 250 W for the P40 and PCIe V100, 300 W for the SXM2 V100, 300 W for the PCIe A100; size the supply with multi-gpu-pcie-and-psu. Warranty and wear history are unknown for any used card. Scam frequency and listing accuracy are unknown here: no source was read on them. Practical rule: pay through a method with buyer protection and ask the seller for a photo of the card’s label.

Can an A100 or H100 be used for local LLMs at home?

Physically, only the PCIe versions are candidates. NVIDIA lists the A100 80GB PCIe as dual-slot air-cooled or single-slot liquid-cooled at 300 W, and the H100 NVL as PCIe dual-slot air-cooled at 350-400 W (configurable). The SXM versions are for server boards; the H100 SXM is up to 700 W (configurable). The pages do not say whether an air-cooled datacenter card is quiet or workable in a desktop case, so that is unknown. Price is unknown. For a workstation card with a warranty, compare rtx-5090-vs-rtx-pro-6000.

Questions

Best GPU for $250: NVIDIA P40 or MI50 32GB?

This page cannot rank them by price, because no price is stated: check current retailer price. On sourced facts, the P40 has 24 GB at 346 GB/s and 250 W, while the MI50 has 32 GiB with bandwidth and power unknown here and is unsupported in current ROCm. The extra 8 GB or so matters only if software runs on it. Speed for both is unknown.

Is the Tesla V100 32GB worth it in 2026?

It has the most memory bandwidth of the budget cards with a published figure: 32 GB of HBM2 at 900 GB/s per NVIDIA’s datasheet. It is passive-cooled, the SXM2 version is not a PCIe card, and its price and software support are unknown here, so the answer depends on those.

Is the MI50 32GB worth it for local LLMs?

It offers 32 GiB per AMD’s ROCm table, but current ROCm lists the MI50 as unsupported, with prebuilt libraries that “are not officially supported and will cause runtime errors”. If you are not willing to work outside official support, it is not worth it. Speed is unknown.

How do I avoid scams buying an MI50?

Confirm the exact memory variant (16 GiB and 32 GiB both exist per AMD), use buyer protection, and ask for photos of the card and its label. These are general precautions. No source read for this page measured how common scams are, so the rate is unknown.

Can an A100 or H100 be used for local LLMs at home?

Only PCIe models are realistic: the A100 80GB PCIe (300 W) or the H100 NVL (350-400 W configurable, 94 GB). The SXM versions need server boards. Their memory is large enough for a 43.1 GB Q4_K_M 70B file, but price, noise and support are unknown here.

Sources read on 2026-10-13

• NVIDIA Tesla P40 datasheet (PDF, Sep 2016): 24 GB, 346 GB/s, PCIe 3.0 x16, dual slot full height, 250 W max. nvidia.com/en-us/data-center/tesla-p40/ returned HTTP 404.
• NVIDIA Tesla V100 datasheet (PDF, Mar 2018): 32GB/16GB HBM2, 900GB/s, PCIe 250 W and SXM2 300 W, NVLink, passive thermal. The V100 web page carried no spec table.
• NVIDIA A100: 80GB HBM2e, 1,935 and 2,039 GB/s, 300 W and 400 W, PCIe and SXM forms.
• NVIDIA H100: 80GB SXM and 94GB NVL, 3.35 and 3.9 TB/s, power and form factor.
• AMD ROCm GPU specs: MI50 at 32 GiB and 16 GiB. AMD’s MI50 driver page returned HTTP 503 on three tries.
• AMD ROCm system requirements: MI50 (gfx906) unsupported, and the definition of that status.
• Qwen2.5-32B card and config.json: 32.5B parameters, 64 layers, 8 KV heads, 40 attention heads, hidden size 5120, bfloat16.
• llama.cpp quantize README: 4.8944 and 8.5008 bits per weight (measured on Llama-3.1-8B), Llama 3.1 70B Q4_K_M 43.1 GB.

Related

• budget-gpu-for-local-llm ranks budget cards.
• rtx-3090-llm is the used consumer 24 GB card.
• amd-gpu-for-local-llm covers AMD software.
• 48gb-vram-llm, multi-gpu-llm and multi-gpu-pcie-and-psu cover two-card builds.
• gpu-vram-list lists VRAM by card.
• methodology, vram-requirements and llm-quantization-explained define the math.
• cuda-out-of-memory-llm covers failures.