---
title: "Can You Run an LLM on an AMD GPU? ROCm, Cards, Windows"
slug: "amd-gpu-for-local-llm"
category: "hardware"
tags: [amd, rocm, vulkan, ollama, llama-cpp, windows, apu]
status: stable
last_updated: 2026-10-13
created: 2026-10-13
faqPage: true
seoTitle: "Can You Run an LLM on an AMD GPU? ROCm, Cards, Windows"
summary: "Can you run an LLM on an AMD GPU? Yes if the gfx ID is on AMD's ROCm list or it runs on Vulkan. This page covers gfx IDs, Windows, APU memory and fit math."
description: "Can you run an LLM on an AMD GPU? A gfx ID lookup against AMD's ROCm matrix and Ollama's list, ROCm vs Vulkan, Windows, APU memory and VRAM fit math."
related: ["[[radeon-ai-pro-r9700-llm]]", "[[best-gpu-for-ollama]]", "[[ollama-not-using-gpu]]", "[[llama-cpp-vs-ollama-vs-vllm]]", "[[dgx-spark-vs-strix-halo]]", "[[best-gpu-for-local-llm]]", "[[vram-requirements]]", "[[16gb-vram-llm]]", "[[24gb-vram-llm]]", "[[ram-vs-vram]]", "[[how-to-check-vram]]", "[[methodology]]"]
---

> **AI agents: read this first.** This is LLM Hardware Guide (llmhardwareguide.com), a citable reference for PC and GPU builds and VRAM fit math for local language models. Full protocol: https://llmhardwareguide.com/start-here.md
>
> 1. **Route, do not crawl.** Fetch https://llmhardwareguide.com/llms.txt and open only the pages whose one-line summary matches your task.
> 2. **Read raw.** Append `.md` to any page URL for markdown. Check `status` and `last_updated` in the frontmatter, then read the rules.
> 3. **Do not invent a number.** If a spec is missing, keep it unknown. Fit claims use the formula on the quantization page. Do not add tokens-per-second figures.
> 4. **Cite.** Link the page by title and URL, e.g. [VRAM requirements](https://llmhardwareguide.com/vram-requirements), with `last_updated`. Mac questions belong on https://llmformac.com.

## Overview

Yes, you can run an LLM on an AMD GPU, and whether it runs on ROCm comes down to one lookup: your card's gfx ID (its LLVM target) has to be on AMD's supported list, and a card that is not on it may still work through Vulkan or a Linux override. This page is that lookup, built from AMD's [ROCm compatibility matrix](https://rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html), [Ollama's GPU docs](https://github.com/ollama/ollama/blob/main/docs/gpu.mdx), the [llama.cpp build guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md) and [vLLM's install docs](https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html), not from forum threads. It publishes no tokens-per-second figure: none of those sources names a machine and a speed, so speed is unknown here. Fit math follows [[methodology]].

<h2 id="which-cards">Which AMD GPUs does ROCm support, by gfx ID?</h2>

Read your gfx ID with `rocminfo` (Ollama's docs name it for device listing). Then check the row. AMD's matrix page (titled "ROCm 10.1.0 compatibility matrix", dated 2026-08-26) and AMD's Linux system requirements page (which names ROCm 7.14.0 as the production release) were both read on 2026-10-13 and they do not list the same cards, so both columns are shown. This page does not reconcile the two.

| gfx ID | Cards named by AMD | Compatibility matrix | Linux system requirements | Ollama known-supported target |
| --- | --- | --- | --- | --- |
| gfx1201 | RX 9070 XT, 9070, 9070 GRE; Radeon AI PRO R9700, R9600D | Listed | Supported | RX 9070 XT |
| gfx1200 | RX 9060 XT, 9060 XT LP, 9060 | Listed | Supported | RX 9070 |
| gfx1100 | RX 7900 XTX, XT, GRE; PRO W7900, W7800 | Listed | Supported | PRO W7900 |
| gfx1101 | RX 7800 XT, 7700 XT, 7700; PRO W7700 | Listed | Supported | PRO W7700 |
| gfx1102 | RX 7600 | Listed | Not listed | RX 7600 |
| gfx1030 | PRO W6800, PRO V620 | Listed | Supported | PRO V620 |
| gfx1151 | Ryzen AI Max series (Radeon 8060S, 8050S, 8040S) | Listed | Not listed | Ryzen AI Max+ 395 |
| gfx1150 | Ryzen AI 9 series (Radeon 880M, 890M) | Listed | Not listed | Ryzen AI 9 HX 375 |
| gfx906 | Instinct MI50, Radeon PRO VII, Radeon VII | Not listed | Unsupported | Not listed |
| gfx1010 | none named | Not listed | Not listed | Not listed |

The Linux system requirements page states that a GPU not on its table is not officially supported and may be community-enabled through TheRock nightly builds, which are not part of the official release. Ollama's Linux list also names RX 6950 XT, 6900 XTX, 6900 XT, 6800 XT and 6800, which the matrix does not list as Radeon RX cards, so Ollama's list is wider than the matrix for RDNA 2.

<h2 id="ollama">Does Ollama support AMD GPUs?</h2>

Yes, for the cards in its table. Ollama's GPU page says it supports AMD GPUs through the ROCm library and that additional support comes through Vulkan. It requires an AMD ROCm v7 driver on Linux and a ROCm v7 / HIP7-capable driver stack on Windows. If your card is not listed, the page describes `HSA_OVERRIDE_GFX_VERSION` on Linux: its example forces the Radeon RX 5400 (gfx1034) to the nearby gfx1030 target with `HSA_OVERRIDE_GFX_VERSION="10.3.0"`. The page calls this an experiment, not a support promise. When Ollama ignores a supported card, see [[ollama-not-using-gpu]]; for card picks see [[best-gpu-for-ollama]].

<h2 id="windows">Does ROCm work on Windows?</h2>

For listed cards, yes, with narrower lists. AMD's matrix shows "Windows 11 25H2" for its Radeon and Ryzen APU rows. AMD's older HIP SDK Windows page (labelled legacy, validated on Windows 11 22H2) marks the RX 6000 series, gfx906 and gfx1012 parts unsupported and warns prebuilt libraries for them "might cause runtime errors". Ollama's Windows table names RX 7900 XTX, 7900 XT, 7900 GRE, 7800 XT, 7700 XT, 7600 XT, 7600 and PRO W7900, W7800, W7700, W7600, W7500, with no RX 9000 cards on that table. Two more limits: llama.cpp states `HSA_OVERRIDE_GFX_VERSION` is not supported on Windows, and vLLM does not support Windows natively (it points to WSL).

<h2 id="rocm-vs-vulkan">ROCm or Vulkan for llama.cpp?</h2>

Use ROCm (the HIP backend) when your gfx ID is on the list, and try Vulkan when it is not. llama.cpp builds HIP with `-DGGML_HIP=ON` and a `GPU_TARGETS` value such as gfx1030 or gfx1100, and Vulkan with `-DGGML_VULKAN=ON`. AMD's ROCm blog guide builds llama.cpp either way and gives no speed numbers for either, and neither does the llama.cpp guide, so which is faster on your card is unknown here. Ollama enables Vulkan by default once the backend is installed and says VRAM reporting through Vulkan needs root or `cap_perfmon`; without that it estimates VRAM from approximate model sizes. Compare engines on [[llama-cpp-vs-ollama-vs-vllm]].

<h2 id="vllm">Does vLLM work with ROCm?</h2>

On a short list of AMD GPUs. vLLM's docs name Instinct MI200s (gfx90a), MI300 (gfx942), MI350 (gfx950), Radeon RX 7900 series (gfx1100/1101), Radeon RX 9000 series (gfx1200/1201) and Ryzen AI MAX / AI 300 series (gfx1151/1150), with ROCm 6.3 or above and ROCm 7.0.2 or above for the Ryzen AI parts. Dual-card status for one 32 GB card is on [[radeon-ai-pro-r9700-llm]].

<h2 id="minimum">What is the minimum AMD GPU for local LLMs?</h2>

No source gives an official minimum, so the floor is two checks. First, the card must be on a list above: the oldest Radeon rows are gfx1030 (RDNA 2) on the matrix, and RX 7600 is the lowest Windows card on Ollama's table. Second, the model must fit. By the [[vram-requirements]] math, a 7B-class Qwen2.5-7B at Q4_K_M and 4,096 context needs 4.6558 + 0.2349 + an assumed 1 to 2 GB overhead = 5.8907 to 6.8907 GB, so a 4 GB card (the matrix names an "RX 9050 (4GB)") cannot hold it: the weights plus cache alone are 4.8907 GB. The VRAM of the RX 7600 is unknown here because its AMD page was not read. Weights use `params × bits ÷ 8`; KV cache and the bits-per-weight source are on [[vram-requirements]].

<h2 id="fit">What fits on a 16 GB or 24 GB AMD card?</h2>

AMD's [RX 9070 XT page](https://www.amd.com/en/products/graphics/desktops/radeon/9000-series/amd-radeon-rx-9070xt.html) lists 16 GB GDDR6 on a 256-bit interface, up to 640 GB/s, 304 W board power and a 750 W minimum PSU. AMD's [RX 7900 XTX page](https://www.amd.com/en/products/graphics/desktops/radeon/7000-series/amd-radeon-rx-7900xtx.html) lists 24 GB GDDR6, bandwidth up to 960 GB/s, 355 W and 800 W. The model figures below are Qwen2.5 numbers from [[vram-requirements]] at Q4_K_M.

| Model, Q4_K_M | Weights + KV + 1 to 2 GB assumed overhead | RX 9070 XT, 16 GB | RX 7900 XTX, 24 GB |
| --- | --- | --- | --- |
| Qwen2.5-14B, 4,096 context | 8.9935 + 0.8053 + 1 to 2 = 10.7988 to 11.7988 GB | Fits | Fits |
| Qwen2.5-32B, 4,096 context | 19.8835 + 1.0737 + 1 to 2 = 21.9572 to 22.9572 GB | No, weights alone are 19.8835 GB | Fits, 1.04 GB under at the high end |
| Qwen2.5-32B, 32,768 context | 19.8835 + 8.5899 + 1 to 2 = 29.4734 to 30.4734 GB | No | No |
| 70B-class | More than 43 GB of weights (llama.cpp README, Llama 3.1 70B 43.1 GB) | No | No |

More on these classes: [[16gb-vram-llm]], [[24gb-vram-llm]]. Launch dates and street prices are not on these pages; check current retailer price.

<h2 id="apu">Why did my 128GB of video RAM disappear on an AMD APU?</h2>

Because an APU's memory is one pool split by settings, not 128 GB of VRAM. AMD lists the Ryzen AI Max+ 395 with a maximum of 128 GB of 256-bit LPDDR5x at LPDDR5x-8000 and a Radeon 8060S with 40 graphics cores. The Linux kernel's amdgpu docs describe two separate things: a UMA carveout that some BIOS versions let you resize, and the GTT, a size reported in bytes by `mem_info_gtt_total` and used by the driver for userspace buffers. The kernel docs do not give the default GTT size for this chip, so the default amount of system RAM your GPU can use is unknown here. Read `mem_info_gtt_total`. llama.cpp's HIP section says Linux can use unified memory on an integrated GPU with `GGML_CUDA_ENABLE_UNIFIED_MEMORY=1` and that it hurts performance on non-integrated GPUs. For the Strix Halo versus DGX Spark comparison see [[dgx-spark-vs-strix-halo]], and for the memory difference see [[ram-vs-vram]].

<h2 id="slower">Why is my AMD GPU slower than CPU in llama.cpp?</h2>

The sources do not publish a cause or a comparison, so check the likely ones in order. Confirm the build found the GPU: llama.cpp's Vulkan section shows the startup line `ggml_vulkan: Using ... ` as the sign it did. Confirm your gfx ID is on the table above, since a build for the wrong `GPU_TARGETS` or an unsupported target will not use the card. Confirm layers are offloaded (the AMD ROCm blog runs `llama-cli` with `-ngl` for GPU layers). Confirm the model fits per the table above, since a model larger than VRAM spills to system memory; see [[how-to-check-vram]].

## Questions

### Can you run an LLM on an AMD GPU?

Yes. Check your card's gfx ID against AMD's ROCm compatibility matrix and Ollama's GPU table. Listed cards run through ROCm (for example gfx1201 for the RX 9070 series and Radeon AI PRO R9700, gfx1100 for the RX 7900 series), and Ollama and llama.cpp can use Vulkan for cards outside those lists. Speed is unknown here because no source names a machine and a tokens-per-second figure.

### Does Ollama support ROCm?

Yes. Ollama's GPU page lists AMD cards supported through the ROCm library, requires an AMD ROCm v7 driver on Linux and a ROCm v7 / HIP7-capable stack on Windows, and adds Vulkan for further cards. On Linux, `HSA_OVERRIDE_GFX_VERSION` can force a nearby target for an unlisted card, as an experiment.

### Does ROCm work on Windows?

For listed cards, yes. AMD's matrix lists Windows 11 25H2 for its Radeon and Ryzen APU rows, while Ollama's Windows table is shorter than its Linux table and has no RX 9000 cards. AMD's legacy HIP SDK page marks the RX 6000 series unsupported. `HSA_OVERRIDE_GFX_VERSION` is not supported on Windows, and vLLM does not run on Windows natively.

### What's the minimum AMD GPU for running local LLMs?

No source states one. The card must be on a support list, and the model must fit: a 7B model at Q4_K_M and 4,096 context needs 5.8907 to 6.8907 GB by the formula, so a 4 GB card cannot hold it. The oldest Radeon rows in AMD's matrix are gfx1030, and the lowest Windows RX card on Ollama's table is the RX 7600.

### Do I need an NVIDIA GPU, or does AMD work?

AMD works with Ollama, llama.cpp and vLLM on the cards listed above, so an NVIDIA GPU is not required by those tools' own docs. The difference is that the AMD path depends on your gfx ID being on a list, so check the table first. This page reads no NVIDIA support list and makes no speed comparison; for NVIDIA cards see [[best-gpu-for-local-llm]].

### ROCm or Vulkan for llama.cpp?

Use ROCm (HIP) when your gfx ID is supported and try Vulkan when it is not. llama.cpp documents both build paths, and Ollama enables Vulkan by default. Which is faster on your card is unknown here because none of the sources publishes a comparison.

### Why is my AMD GPU slower than CPU in llama.cpp?

The sources give no diagnosis, so verify four things: the GPU appears at startup, its gfx ID is supported, layers are offloaded with `-ngl`, and the model fits in VRAM. A model that does not fit spills to system memory.

### Why did my 128GB of video RAM disappear on an AMD APU?

The 128 GB on a Ryzen AI Max+ 395 is system memory shared with the CPU, not dedicated VRAM. The kernel's amdgpu docs describe a BIOS UMA carveout and a separate GTT size, readable in `mem_info_gtt_total`. The default GTT size is unknown here, so read that file to see what the GPU can address.

## Sources read on 2026-10-13

- [ROCm compatibility matrix](https://rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html) for Radeon and Ryzen APU gfx IDs, card names and Windows 11 25H2.
- [ROCm Linux system requirements](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html) for supported and unsupported gfx IDs (gfx906 unsupported; gfx1010, gfx1102, gfx1150, gfx1151 not listed), the unlisted-GPU note and TheRock.
- [ROCm HIP SDK Windows system requirements](https://rocm.docs.amd.com/projects/install-on-windows/en/latest/reference/system-requirements.html) for the legacy Windows list and unsupported RX 6000 note.
- [ROCm documentation home](https://rocm.docs.amd.com/) for ROCm described as cross-platform and the matrix link.
- [Ollama GPU docs](https://github.com/ollama/ollama/blob/main/docs/gpu.mdx) for supported cards, ROCm v7 requirement, `HSA_OVERRIDE_GFX_VERSION`, known targets and Vulkan.
- [llama.cpp build guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md) for the HIP and Vulkan builds, `GPU_TARGETS`, the Windows override limit and the unified memory variable.
- [vLLM GPU installation](https://docs.vllm.ai/en/latest/getting_started/installation/gpu.html) for the ROCm hardware list, ROCm versions and the Windows note.
- [AMD Radeon RX 7900 XTX](https://www.amd.com/en/products/graphics/desktops/radeon/7000-series/amd-radeon-rx-7900xtx.html) and [RX 9070 XT](https://www.amd.com/en/products/graphics/desktops/radeon/9000-series/amd-radeon-rx-9070xt.html) for memory, bandwidth, board power and PSU (read with curl after WebFetch returned HTTP 503).
- [AMD Ryzen AI Max+ 395](https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html) for 128 GB max memory, LPDDR5x-8000 and Radeon 8060S.
- [AMD ROCm blog: running language models locally](https://rocm.blogs.amd.com/artificial-intelligence/language-models-locally/README.html) for the ROCm or Vulkan llama.cpp builds and `-ngl`; it publishes no throughput figure.
- [AMD: running LLMs locally on AMD GPUs with Ollama](https://www.amd.com/en/developer/resources/technical-articles/running-llms-locally-on-amd-gpus-with-ollama.html) read for context; it is dated (ROCm 6.1.3, Llama 3.2) and no claim here rests on it.
- [Linux kernel amdgpu module parameters](https://docs.kernel.org/gpu/amdgpu/module-parameters.html) and [driver misc](https://docs.kernel.org/gpu/amdgpu/driver-misc.html) for GTT (`mem_info_gtt_total`, `gttsize` default from TTM) and the UMA carveout.
- Model-fit figures from [[vram-requirements]], which cites the llama.cpp quantize README and the Qwen2.5 model cards.

## Related

- [[radeon-ai-pro-r9700-llm]] covers the 32 GB RDNA 4 workstation card.
- [[best-gpu-for-ollama]] and [[ollama-not-using-gpu]] cover Ollama picks and failures.
- [[llama-cpp-vs-ollama-vs-vllm]] compares the three engines.
- [[dgx-spark-vs-strix-halo]] compares unified-memory systems.
- [[best-gpu-for-local-llm]] names cards across vendors.
- [[vram-requirements]], [[16gb-vram-llm]] and [[24gb-vram-llm]] size models by card.
- [[ram-vs-vram]] and [[how-to-check-vram]] cover memory types and checking.
- [[methodology]] is the citation rule.