Overview
The Radeon AI PRO R9700 is a 32 GB GDDR6 workstation card from AMD with a 256-bit memory interface and 640 GB/s peak bandwidth, which makes it a 32 GB option for local LLMs without NVIDIA’s CUDA. Whether it is a good buy depends on software: AMD lists it as supported by ROCm, but a dual-card vLLM report was still open when read on 2026-10-13. This page gives the vendor specs, the fit math and the caveats. It publishes no tokens-per-second figure, because AMD’s pages publish none and this site does not estimate them.
How much VRAM does the Radeon AI PRO R9700 have?
AMD’s R9700 page lists 32 GB of GDDR6, a 256-bit memory interface, 64 MB of Infinity Cache, 640 GB/s peak memory bandwidth, and ECC support on Linux only. The GPU is RDNA 4 with 64 compute units, 128 AI accelerators and 4096 stream processors. The board bus is PCIe 5.0 x16, the cooler is active, and the card is double slot. It needs a 12V-2x6 power connector. Total board power is 300 W with a 750 W minimum PSU recommendation.
AMD’s page also lists peak matrix figures, for example 191 TFLOPs FP16 matrix and 383 TOPs INT8 matrix. These are peak ratings, not LLM speed, and they are not converted into one here. Launch date and price are not on the AMD page: both unknown. Check current retailer price.
What is the difference between the R9700, R9700S and R9600D?
All three AMD pages list 32 GB GDDR6, 256-bit, 640 GB/s and PCIe 5.0 x16. They differ in power, cooling and compute.
| Spec (AMD page) | R9700 | R9700S | R9600D |
|---|---|---|---|
| Memory | 32 GB GDDR6 | 32 GB GDDR6 | 32 GB GDDR6 |
| Peak bandwidth | 640 GB/s | 640 GB/s | 640 GB/s |
| Compute units | 64 | 64 | 48 |
| AI accelerators | 128 | 128 | 96 |
| Total board power | 300 W | 300 W | 150 W |
| Minimum PSU recommendation | 750 W | 750 W | 450 W |
| Cooling | Active | Passive | Passive |
| Board width | Double slot | Double slot | Single slot |
| Peak FP16 matrix | 191 TFLOPs | 191 TFLOPs | 99 TFLOPs |
Sources: R9700, R9700S, R9600D. A passive card relies on chassis airflow; the pages do not say which systems the S and D variants ship in, so that is unknown. Same memory size and bandwidth means the weight-fit math below is identical for all three. Compute-bound work such as long prompt processing differs, and no source here quantifies by how much.
What fits in 32 GB?
Using the formulas on methodology and vram-requirements: weights_GB = params x bits / 8 / 1e9, plus KV cache, plus an assumed 1 to 2 GB overhead (an assumption, not a measurement). The Qwen figures below come from vram-requirements, which cites the Qwen2.5 model cards and llama.cpp’s quantize README.
- Qwen2.5-32B at Q4_K_M, 4,096 context: 19.8835 + 1.0737 + 1 to 2 = 21.9572 to 22.9572 GB. This fits in 32 GB with room to spare.
- Qwen2.5-32B at Q4_K_M, 32,768 context: 19.8835 + 8.5899 + 1 to 2 = 29.4734 to 30.4734 GB. This is a fit by arithmetic with 1.5 to 2.5 GB left, so a narrow one.
- Qwen2.5-32B at Q8_0: weights alone are 34.5345 GB, over 32 GB on one card.
- Qwen2.5-72B at Q4_K_M, 4,096 context: 44.4779 + 1.3422 + 1 to 2 = 46.8201 to 47.8201 GB. This does not fit one 32 GB card.
Bandwidth matters for decode speed in general, but AMD publishes only the 640 GB/s peak; any speed from it would be a guess, so speed is unknown. For the same capacity class on NVIDIA, NVIDIA’s RTX 5090 page lists 32 GB GDDR7 and 575 W total graphics power (no memory bandwidth figure was on the page read).
Does ROCm support the R9700?
Yes, on paper. The ROCm system requirements page lists “AMD Radeon AI PRO R9700” as RDNA4, LLVM target gfx1201, with a supported mark. Its footnote says the Radeon PRO and Radeon GPUs support only Ubuntu 24.04.4, Ubuntu 22.04.5, RHEL 10.1 and RHEL 9.7. The page notes it has moved and points to the ROCm compatibility matrix for the latest status, so check that matrix before buying. AMD’s product pages list Windows 10, Windows 11 and Linux x86 64-bit as OS support for the card, but the ROCm page is the one that governs the ROCm toolchain. Which inference engines work well on it is covered in llama-cpp-vs-ollama-vs-vllm and amd-gpu-for-local-llm.
Does vLLM tensor parallel work on two R9700 cards (dual R9700)?
It was reported broken and still open on 2026-10-13. vLLM issue #40980, “TP=2 deadlock on dual AMD R9700 (gfx1201/RDNA4)”, was opened Apr 27, 2026 and is labelled bug and rocm. The reporter says both GPUs sit at 100% utilization with no active requests and inference hangs at tensor parallel size 2, while TP=1 works. Reported stack: vLLM v0.19.1+rocm721, PyTorch 2.11.0, ROCm 7.2.1, RCCL 2.27.7. Later comments say TP=4 on four R9700 cards is also affected.
Workarounds in the thread: setting NCCL_PROTO=Simple (suggested by a maintainer on May 8 and confirmed by several commenters, one of whom also reported slower token generation on 0.19 and later), downgrading RCCL, or using the vllm-openai-rocm:v0.18.1 image. A commenter said ROCm PRs #6966 and #7183 were merged and should fix it in the next RCCL release; the thread does not state the outcome of a July 23 test on v0.25.1. Treat dual-card vLLM as unverified until you check the issue. The reporter’s hardware line says 16 GB VRAM each, which does not match AMD’s 32 GB; this page relies on AMD’s figure. Two cards give 2 x 32 = 64 GB of total memory, but tensor parallel splits weights across cards, so it is not one 64 GB pool for every engine and every setting; per-card overhead is unknown. See multi-gpu-llm for the general trade-offs. No dual-R9700 benchmark is cited here.
R9700 or RTX 5090 for local LLMs?
On vendor pages alone, both are 32 GB cards, so the same model files fit on both. The differences are what the pages state: the R9700 is 300 W with a 750 W minimum PSU recommendation; the RTX 5090 is 575 W total graphics power. The R9700 runs ROCm and the 5090 runs CUDA, and which has more tooling support is not something a vendor spec page answers. Speed is unknown for both. See rtx-5090-llm, 32gb-vram-llm and arc-pro-b70-vs-r9700 for the other 32 GB comparisons.
Questions
Is the Radeon AI PRO R9700 good for local LLMs?
It is a good fit on capacity: 32 GB of GDDR6 at 640 GB/s peak per AMD holds a 32B model at Q4_K_M with room for context. The risks are software: ROCm support is limited to specific Linux distributions per AMD, and dual-card vLLM had an open deadlock report on 2026-10-13. Speed is unknown because no source cited here measures it.
How much VRAM does the Radeon AI PRO R9700 have?
32 GB of GDDR6 on a 256-bit interface with 640 GB/s peak memory bandwidth, according to AMD’s product page.
What is the difference between the R9700, R9700S and R9600D?
All three have 32 GB GDDR6 at 640 GB/s. The R9700 is 300 W, active cooling, 64 compute units. The R9700S is 300 W, passive, 64 compute units. The R9600D is 150 W, passive, single slot, 48 compute units. Launch dates and prices are unknown from AMD’s pages.
Does vLLM tensor parallel work on two R9700 cards?
Not reliably as of the issue read on 2026-10-13. vLLM issue #40980 reports a TP=2 deadlock on dual R9700, with TP=1 working, and lists workarounds such as NCCL_PROTO=Simple. The issue was still open.
R9700 or RTX 5090 for local LLMs?
Both list 32 GB, so capacity is equal. The R9700 is rated 300 W against 575 W for the RTX 5090 on the vendor pages; speed and price are unknown here, so check current retailer price and published benchmarks that name the machine.
Sources read on 2026-10-13
- AMD Radeon AI PRO R9700: memory, bandwidth, compute units, power, PSU, cooling, board width, OS support.
- AMD Radeon AI PRO R9700S: same fields; passive cooling.
- AMD Radeon AI PRO R9600D: same fields; 150 W, 48 CUs, single slot.
- ROCm system requirements: R9700 row (gfx1201), supported OS footnote.
- vLLM issue #40980: TP=2 deadlock report, versions, workarounds, status.
- NVIDIA RTX 5090: 32 GB GDDR7, 575 W.
- llama.cpp quantize README and Qwen2.5 model cards, via the figures on vram-requirements.