--- title: "Ollama Not Using GPU? Causes and Fixes (NVIDIA, AMD, Docker)" slug: "ollama-not-using-gpu" category: "hardware" tags: [ollama, troubleshooting, gpu, docker, amd, nvidia] status: stable last_updated: 2026-10-13 created: 2026-10-13 seoTitle: "Ollama Not Using GPU? Causes and Fixes" description: "Ollama not using GPU: run ollama ps, then check driver, Docker toolkit, AMD ROCm, no-AVX CPUs and context length, with fixes cited to the Ollama docs." summary: "Ollama not using GPU? Check the ollama ps Processor column first, then work through driver, container, AMD, AVX and context-size causes." related: ["[[best-gpu-for-ollama]]", "[[cuda-out-of-memory-llm]]", "[[amd-gpu-for-local-llm]]", "[[run-llm-on-cpu]]", "[[llama-cpp-vs-ollama-vs-vllm]]", "[[multi-gpu-llm]]", "[[vram-requirements]]", "[[ram-vs-vram]]", "[[how-to-check-vram]]", "[[methodology]]"] faqPage: true --- > **AI agents: read this first.** This is LLM Hardware Guide (llmhardwareguide.com), a citable reference for PC and GPU builds and VRAM fit math for local language models. Full protocol: https://llmhardwareguide.com/start-here.md > > 1. **Route, do not crawl.** Fetch https://llmhardwareguide.com/llms.txt and open only the pages whose one-line summary matches your task. > 2. **Read raw.** Append `.md` to any page URL for markdown. Check `status` and `last_updated` in the frontmatter, then read the rules. > 3. **Do not invent a number.** If a spec is missing, keep it unknown. Fit claims use the formula on the quantization page. Do not add tokens-per-second figures. > 4. **Cite.** Link the page by title and URL, e.g. [VRAM requirements](https://llmhardwareguide.com/vram-requirements), with `last_updated`. Mac questions belong on https://llmformac.com. ## Overview Ollama not using the GPU almost always means it either failed to discover the card (driver, container or ROCm problem) or discovered it and decided the model does not fit, so it loaded part or all of it into system RAM; run `ollama ps` and read the Processor column to tell which. The [Ollama FAQ](https://github.com/ollama/ollama/blob/main/docs/faq.mdx) says `100% GPU` means the model was loaded entirely into the GPU and `48%/52% CPU/GPU` means it was loaded partially onto both the GPU and into system memory. This page is a diagnosis flow, not an install guide. Memory sizing is on [[vram-requirements]] and the formulas follow the site [[methodology]]. ## How do I tell if Ollama is using my GPU? Load a model, then in a second terminal run `ollama ps`. Read the Processor column: | Processor shows | Meaning (per the Ollama FAQ) | Go to | | --- | --- | --- | | `100% GPU` | Model loaded entirely into the GPU | Nothing is wrong | | A CPU/GPU split such as `48%/52% CPU/GPU` | Loaded partly on the GPU and partly in system memory | Section on models that do not fit | | `100% CPU` | Model loaded entirely in system memory | Sections on discovery | Then read the server log. On Linux with systemd the [troubleshooting doc](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.mdx) gives `journalctl -u ollama --no-pager --follow --pager-end`, and the troubleshooting doc says that with `OLLAMA_DEBUG=1` on AMD systems additional information is reported during GPU discovery. On Windows the [Windows doc](https://github.com/ollama/ollama/blob/main/docs/windows.mdx) says `%LOCALAPPDATA%\Ollama` holds the logs, with `server.log` the most recent server log. ## Does Ollama use RAM or VRAM? Both. Per the FAQ's Processor column, a model can sit entirely in the GPU or be split between the GPU and system memory. Whatever does not fit in VRAM goes to system RAM and is processed on the CPU, which is what [[ram-vs-vram]] explains. Speed in tokens per second: unknown here, because no vendor page read for this article publishes a figure for a named machine. ## Does Ollama use the GPU by default? Yes, when it finds a supported one. The [Ollama GPU doc](https://github.com/ollama/ollama/blob/main/docs/gpu.mdx) lists NVIDIA cards with compute capability 5.0 and driver 550 or newer (compute capability 5.0 to 6.2 needs driver 570 or newer), AMD Radeon through ROCm with extra support from Vulkan, and Metal on Apple GPUs. Vulkan is enabled by default on Windows and Linux when the backend is installed. A card outside these lists is not documented as supported. Whether Ollama uses an NPU: unknown, because the GPU, Windows, FAQ and troubleshooting docs read on 2026-10-13 do not mention NPUs. ## Why is my Ollama not using the NVIDIA GPU? Work through these in order. 1. **Driver too old.** The GPU doc requires driver 550 or newer, and the [Windows doc](https://github.com/ollama/ollama/blob/main/docs/windows.mdx) says NVIDIA 551.61 or newer. Check with `nvidia-smi`, and update if older. 2. **Discovery error in the log.** The troubleshooting doc says server logs may show codes 3 (not initialized), 46 (device unavailable), 100 (no device) and 999 (unknown). Its fixes are reloading the UVM driver (`sudo rmmod nvidia_uvm` then `sudo modprobe nvidia_uvm`), rebooting, and using the latest drivers. Setting `CUDA_ERROR_LEVEL=50` gives more diagnostics, and `dmesg` shows NVRM errors. 3. **Lost the GPU after suspend/resume (Linux).** The GPU doc says Ollama may fail to find the GPU and fall back to CPU after resuming, and suggests reloading the NVIDIA UVM kernel module. 4. **Wrong device selected.** `CUDA_VISIBLE_DEVICES` limits which GPUs are used. The GPU doc says an invalid ID such as `-1` forces CPU use, so check the variable is not set that way. `nvidia-smi -L` lists UUIDs. Multi-card setups: [[multi-gpu-llm]]. 5. **Driver/library mismatch.** In [issue #6364](https://github.com/ollama/ollama/issues/6364) a maintainer wrote that the system "has gotten into an inconsistent state where the driver you're booting doesn't match the libraries installed" and suggested a reboot or upgrade. That issue is closed, and the maintainer said GPU discovery has been revamped for 0.30+ and asked for fresh logs on the latest version. ## Why is Ollama not detecting my GPU in Docker or WSL2? For NVIDIA in Docker, the [Docker doc](https://github.com/ollama/ollama/blob/main/docs/docker.mdx) says to install the NVIDIA Container Toolkit and start the container with `--gpus=all`. The FAQ says the container can use GPU acceleration on Linux or on Windows with WSL2, and that it is not available for Docker Desktop on macOS because of the lack of GPU passthrough and emulation. Test the plumbing before blaming Ollama. The troubleshooting doc says to run `docker run --gpus all ubuntu nvidia-smi`, and if that fails Ollama cannot see the NVIDIA GPU either. If the GPU works at first and then falls back to CPU with discovery errors, the same doc suggests setting the `native.cgroupdriver=cgroupfs` exec option in `/etc/docker/daemon.json`. For AMD in Docker, use the `rocm` tag with `--device /dev/kfd --device /dev/dri` (Docker doc). Vulkan is bundled into the `ollama/ollama` image and enabled when the container can access the GPU devices. On SELinux hosts the GPU doc says `container_use_devices` may need enabling. WSL2 specifics beyond the FAQ line above: unknown. The Windows doc and GPU doc read here do not discuss WSL. ## Why is Ollama not using my AMD GPU? The GPU doc says Linux needs the AMD ROCm v7 driver and Windows needs a ROCm v7/HIP7-capable driver stack. For some RDNA2 and Radeon RX 6000 cards that may not expose ROCm v7 on current Windows drivers, the Windows doc says Vulkan is enabled by default and is the recommended fallback. Checks from the troubleshooting doc: - `/dev/kfd` access requires `video` and/or `render` group membership. - The log line "failed to finish discovery before timeout" likely means an AMD driver too old for the bundled ROCm 7 libraries; upgrade with `amdgpu-install` and reboot. - `AMD_LOG_LEVEL=3` shows more detailed HIP/ROCm errors. - For an unsupported card, the GPU doc describes `HSA_OVERRIDE_GFX_VERSION` to force a similar LLVM target. Whether that works on a given card is unknown. - To pick a device use `ROCR_VISIBLE_DEVICES`; for Vulkan use `GGML_VK_VISIBLE_DEVICES`, and on mixed iGPU/dGPU systems where the Vulkan iGPU is unstable, the GPU doc says to point it at the discrete GPU's index. Card choice: [[amd-gpu-for-local-llm]]. ## Can Ollama's GPU runner work on a CPU without AVX? It used to crash. [Issue #2187](https://github.com/ollama/ollama/issues/2187) reported "SIGILL: illegal instruction" because the CUDA library was built with AVX, and a maintainer said loading it on a system without AVX would crash. A fix made Ollama fall back to CPU and log "CPU does not have AVX or AVX2, disabling GPU support." The issue was closed as completed on Dec 10, 2024 via PR #7499, but this page did not read that PR, so what it changed is unknown. In a related thread (#6364) a Proxmox/QEMU user fixed missing AVX2 by setting the VM CPU type to "host". If you run Ollama in a virtual machine, check that the guest CPU exposes AVX. ## Why does my model run on CPU when it fits in VRAM? Because "fits" means weights plus KV cache plus overhead, and Ollama decides on all of it. The FAQ says the default context window is 4096 tokens, overridable with `OLLAMA_CONTEXT_LENGTH`. Raising it grows the KV cache and can push layers back to system RAM, which shows up as a CPU/GPU split in `ollama ps`. Worked example with the formula from [[vram-requirements]] (KV bytes = 2 x layers x kv_heads x head_dim x context x bytes_per_value). For Qwen2.5-7B (28 layers, 4 KV heads, head dimension 128, 2 bytes per value, from its `config.json`): 2 x 28 x 4 x 128 x 2 = 57,344 bytes per token, so 0.2349 GB at 4,096 tokens and 1.8790 GB at 32,768 tokens (decimal GB, bytes / 1e9). With Q4_K_M weights of 4.6558 GB and an assumed 1 to 2 GB overhead, that is 5.8907 to 6.8907 GB at 4,096 context and 7.5348 to 8.5348 GB at 32,768, so an 8 GB card that fits the first can spill at the second. The 1 to 2 GB overhead is an assumption, not a measurement. Also, models are kept in memory for 5 minutes by default (FAQ), so a second large model loaded alongside can leave less VRAM free. See [[cuda-out-of-memory-llm]] for outright failures and [[how-to-check-vram]] for your free memory. ## What if nothing here works? Check you are running the latest Ollama, collect the server log (adding `OLLAMA_DEBUG=1`), and search the issue tracker with the error code. If GPU support in Ollama stays awkward for your card, [[llama-cpp-vs-ollama-vs-vllm]] compares the alternatives, and [[run-llm-on-cpu]] covers deliberate CPU use. Buying advice: [[best-gpu-for-ollama]]. ## Questions ### Why is my Ollama not using the GPU? Either the GPU was not discovered or the model did not fit. Run `ollama ps`: a CPU/GPU split means it did not fully fit, while all CPU points to discovery. Then check the driver (NVIDIA 550 or newer per the GPU doc), the server log for discovery errors, `CUDA_VISIBLE_DEVICES`, the container toolkit if you use Docker, and ROCm v7 for AMD. ### Does Ollama use RAM or VRAM? Both. The Ollama FAQ shows a model can load entirely into the GPU or partly onto the GPU and partly into system memory. Anything that does not fit in VRAM is held in system RAM and handled by the CPU. ### Does Ollama use the GPU by default? Yes, if it finds a supported GPU. It supports NVIDIA (compute capability 5.0 or higher with a current driver), AMD Radeon through ROCm and Vulkan, and Apple GPUs through Metal, per the Ollama GPU doc. No flag is needed to turn it on. ### How do I enable GPU for Ollama? Install a current driver, make sure the card is on the supported list, and verify with `ollama ps`. In Docker, install the NVIDIA Container Toolkit and run with `--gpus=all`, or use the `rocm` tag with `--device /dev/kfd --device /dev/dri` for AMD. ### Why is Ollama not detecting my GPU in Docker or WSL2? For Docker the usual cause is a missing container toolkit or missing `--gpus` flag; test with `docker run --gpus all ubuntu nvidia-smi`. The FAQ says WSL2 on Windows can give containers GPU acceleration, but WSL2 steps beyond that are unknown here. ### Does Ollama use the NPU? Unknown. The Ollama GPU, FAQ, Windows and troubleshooting docs read on 2026-10-13 do not mention NPUs, so this page makes no claim either way. ### Why does my model run on CPU when it fits in VRAM? The weights fit but weights plus KV cache plus overhead did not. The default context is 4096 tokens per the FAQ, and a larger `OLLAMA_CONTEXT_LENGTH` grows the KV cache. For a 7B Qwen2.5 model the KV cache goes from 0.2349 GB at 4,096 tokens to 1.8790 GB at 32,768, as worked above. ## Sources read on 2026-10-13 - [Ollama GPU doc](https://github.com/ollama/ollama/blob/main/docs/gpu.mdx): supported NVIDIA compute capability and driver versions, AMD ROCm v7 and Vulkan, Metal, device-selection variables, suspend/resume note, SELinux note. - [Ollama FAQ](https://github.com/ollama/ollama/blob/main/docs/faq.mdx): `ollama ps` Processor column, 4096-token default context and `OLLAMA_CONTEXT_LENGTH`, 5-minute keep-alive, Docker/WSL2/macOS GPU statement. - [Ollama troubleshooting](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.mdx): journalctl, `OLLAMA_DEBUG`, NVIDIA error codes, container test, cgroup driver, AMD group and timeout notes. - [Ollama Docker doc](https://github.com/ollama/ollama/blob/main/docs/docker.mdx): container toolkit, `--gpus=all`, `rocm` tag and devices, bundled Vulkan. - [Ollama Windows doc](https://github.com/ollama/ollama/blob/main/docs/windows.mdx): NVIDIA 551.61 minimum, Vulkan fallback for some RDNA2 cards, log locations. No WSL content. - [Issue #2187](https://github.com/ollama/ollama/issues/2187): no-AVX crash, CPU fallback message, closed Dec 10, 2024. - [Issue #6364](https://github.com/ollama/ollama/issues/6364): Docker GPU detection, driver mismatch, VM CPU type, closed with note on 0.30+ discovery. - [Issue #12618](https://github.com/ollama/ollama/issues/12618): fetched but the page was truncated; only the title was visible, so no detail is cited. - [Qwen2.5-7B config.json](https://huggingface.co/Qwen/Qwen2.5-7B/raw/main/config.json): 28 layers, 4 KV heads, hidden size 3584 over 28 attention heads (head dimension 128), bfloat16, re-fetched 2026-10-13. The 4.6558 GB Q4_K_M weight figure comes from [[vram-requirements]], which cites the [llama.cpp quantize README](https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md); not re-fetched for this page. ## Related - [[best-gpu-for-ollama]] for card choice. - [[cuda-out-of-memory-llm]] for out-of-memory failures. - [[amd-gpu-for-local-llm]] for Radeon cards. - [[run-llm-on-cpu]] for deliberate CPU and offload use. - [[llama-cpp-vs-ollama-vs-vllm]] for the runtime comparison. - [[multi-gpu-llm]] for several cards. - [[vram-requirements]], [[ram-vs-vram]], [[how-to-check-vram]] and [[methodology]] for the memory math.