To run Ollama in Docker, pull the official ollama/ollama image from Docker Hub and start it with one command: docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama. Add --gpus=all for an NVIDIA card, or switch to the ollama/ollama:rocm tag with /dev/kfd and /dev/dri for AMD, then run a model with docker exec -it ollama ollama run llama3.2.
That plain command starts, answers on port 11434, and on most machines it is also quietly running every model on the CPU. A container cannot see a GPU unless the host runtime hands one over, and Ollama does not error when that hand-over is missing. It just runs slow.
Install Ollama in Docker, step by step
The flags matter, since every later variant builds on them. From the official Docker docs, on any host with Docker Engine or Docker Desktop:
- Pull the image:
docker pull ollama/ollama. - Start the server:
docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama. - Download and chat with a model inside the container:
docker exec -it ollama ollama run llama3.2. - Check the API from the host:
curl http://localhost:11434/api/versionreturns the running version as JSON, per the API reference.
What the run line does:
-v ollama:/root/.ollamais a named volume for the model store. Without it, every pulled model dies with the container. The defaultllama3.2tag is 2.0 GB per its library page, and larger models run to tens of gigabytes, so put the volume somewhere with room.-p 11434:11434publishes the API on every host interface. More on that below.ollama/ollamais the Docker Hub image. Its tags and sizes are in the next section.
The same pull and prompt work over the API from the host, which is how applications will talk to the container:
curl http://localhost:11434/api/pull -d '{"model": "llama3.2"}'
curl http://localhost:11434/api/generate -d '{"model": "llama3.2", "prompt": "Why is the sky blue?"}'Ollama API examples has the chat, streaming and Python versions. What to expect from a CPU-only container is a memory-bandwidth question, not a Docker one. Ollama without a GPU covers the numbers. If the box is a headless server and you are still choosing a runner, see Ollama vs LM Studio for headless serving.
The ollama/ollama image on Docker Hub: tags and sizes
Ollama publishes its own image as ollama/ollama, registered on Docker Hub in June 2023 and past 184 million pulls when its repository API was checked on 9 October 2026. Every release gets a plain version tag and a -rocm twin; latest and rocm track the newest release, and -rc tags are release candidates. Figures from the Docker Hub tag API on that date:
| Tag | Platforms | Compressed size | Backends |
|---|---|---|---|
latest, 0.40.2 | linux/amd64, linux/arm64 | 3.78 GB (amd64), 2.81 GB (arm64) | CPU, NVIDIA CUDA, Vulkan |
rocm, 0.40.2-rocm | linux/amd64 only | 1.45 GB | AMD ROCm |
0.40.2-rc0 and other -rc tags | same as their release | about the same | pre-release testing only |
docker pull ollama/ollama:0.40.2 # CPU, NVIDIA, Vulkan
docker pull ollama/ollama:0.40.2-rocm # AMD ROCmlatest moves fast: two stable releases landed on 7 and 8 October 2026 (0.40.1 and 0.40.2), so pin a version tag in anything you expect to keep running. The final stage of Ollama’s Dockerfile is FROM ubuntu:24.04, sets OLLAMA_HOST=0.0.0.0:11434, runs /bin/ollama serve, and declares no VOLUME, which is why the -v flag is your job.
Is there an Alpine Ollama image?
No. A Docker Hub tag search for alpine on ollama/ollama returns zero of the repository’s 1,232 tags, and the runtime base is Ubuntu 24.04. Alpine’s own ollama package sits only in the edge community repository, at 0.17.7-r1 built on 20 March 2026, with nothing in the v3.22 stable branch.
That is far behind the 0.40.2 the official image ships, so use the official image unless you specifically need Alpine and can live with a much older Ollama.
NVIDIA GPU passthrough
Docker cannot see a GPU by itself. The NVIDIA Container Toolkit installs a runtime hook that maps the driver and device nodes into the container. The guide was at toolkit 1.20.1 when checked in October 2026. Install the host driver first, then on Debian and Ubuntu:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart dockerFedora and RHEL take the .repo file from the same guide and dnf install nvidia-container-toolkit, then the same nvidia-ctk and restart lines. Confirm the runtime works before involving Ollama, using the check from Docker’s GPU docs:
docker run -it --rm --gpus all ubuntu nvidia-smiIf that prints your card, start Ollama with the same flag:
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama--gpus device=0 or --gpus '"device=0,2"' restricts the container to particular cards. Ollama also honours CUDA_VISIBLE_DEVICES per its GPU docs, which list compute capability 5.0 and driver version 550 as the floor, with cards from compute capability 5.0 to 6.2 needing driver 570 or newer.
Two platforms need a different answer. On macOS, Ollama’s own Docker image announcement says to run it as a standalone application, because Docker Desktop does not pass the GPU through, so a containerised Ollama on a Mac is CPU-only; Ollama on Apple Silicon covers the native route. On Windows, Docker Desktop’s GPU support exists only on the WSL 2 backend with an NVIDIA card and a driver that supports GPU paravirtualization.
nvidia-ctk cdi generate: GPUs by name with CDI
sudo nvidia-ctk cdi generate --output=/var/run/cdi/nvidia.yaml writes a Container Device Interface (CDI) spec naming every NVIDIA GPU on the host, so a container runtime can attach one by name, such as nvidia.com/gpu=all, instead of going through the --gpus hook. Podman needs it, and Docker accepts it natively. The steps, from NVIDIA’s CDI guide:
- Generate the spec (toolkit 1.12.0 or later) and list the device names it defines:
sudo nvidia-ctk cdi generate --output=/var/run/cdi/nvidia.yaml
nvidia-ctk cdi list- Start Ollama with a CDI device instead of
--gpus. Docker:
docker run -d --device nvidia.com/gpu=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollamaPodman 4.1.0 or later on Fedora and RHEL:
podman run -d --device nvidia.com/gpu=all --security-opt=label=disable -v ollama:/root/.ollama -p 11434:11434 --name ollama docker.io/ollama/ollama- Keep the spec current. From toolkit 1.18.0, the
nvidia-cdi-refreshsystemd service regenerates it on toolkit or driver install and upgrade, and at boot, but not after driver removal or MIG reconfiguration. Runsudo systemctl restart nvidia-cdi-refresh.serviceafter either.
nvidia.com/gpu=0 picks one card and nvidia.com/gpu=1:0 the first MIG device on GPU 1. If a name will not resolve, nvidia-ctk --debug cdi list shows spec-loading errors. The Ollama image sets NVIDIA_VISIBLE_DEVICES=all, which NVIDIA’s guide says conflicts with a leftover /usr/share/containers/oci/hooks.d/oci-nvidia-hook.json; delete that file on Podman hosts that still have it.
The docker run reference says CDI is enabled by default since Docker Engine 28.3.0, for Linux containers only. Per the dockerd reference, specs load from /etc/cdi and /var/run/cdi, the switch is the cdi key under features in daemon.json, and docker info shows the directories in use. Compose takes the same name under devices: as - "nvidia.com/gpu=all", per the services reference.
AMD GPU passthrough with ROCm
AMD cards use a separate image tag and device mounts instead of the --gpus flag:
docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocmPer AMD’s container docs, /dev/kfd is the compute interface shared by all GPUs, /dev/dri holds the per-GPU render nodes, and the host needs the amdgpu-dkms kernel driver plus Docker Engine. Ollama’s GPU docs want the ROCm v7 driver, installed with amdgpu-install; the image is built against ROCm 7.2.1 and is linux/amd64 only. Cards those docs list as supported on Linux:
| Family | Supported cards |
|---|---|
| Radeon RX 9000 | six models, 9060 through 9070 XT |
| Radeon RX 7000 | eight models, 7600 through 7900 XTX |
| Radeon RX 6000 | 6800, 6800 XT, 6900 XT, 6900 XTX, 6950 XT |
| Radeon PRO and AI PRO | W7500 through W7900, W6800, W6800X, W6800X Duo, W6900X, V620, R9700, R9600D |
| Ryzen AI | Max+ 395, Max 390, Max 385, six Ryzen AI 9 chips (365 to HX 475) |
| Instinct | MI100, MI210, MI250, MI250X, MI300A, MI300X, MI350X |
A card outside that list can borrow the nearest LLVM target with HSA_OVERRIDE_GFX_VERSION. The docs’ example is the Radeon RX 5400, a gfx1034 part that runs as gfx1030:
docker run -d --device /dev/kfd --device /dev/dri -e HSA_OVERRIDE_GFX_VERSION=10.3.0 \
-v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocmMixed cards take per-device variables such as HSA_OVERRIDE_GFX_VERSION_0. To give the container one GPU, pass a single render node next to /dev/kfd (--device /dev/dri/renderD128; ls /dev/dri/render* lists them), or set ROCR_VISIBLE_DEVICES to a UUID from rocminfo. On SELinux hosts the GPU docs recommend sudo setsebool container_use_devices=1 when the container cannot open the devices. The default image also bundles Vulkan, enabled whenever the container can reach the GPU device nodes, with OLLAMA_VULKAN=0 to turn it off. Still choosing a card? The GPU buying guide compares them by VRAM and bandwidth.
As a Compose service, using the HOST:CONTAINER device form from the Compose reference:
services:
ollama:
image: ollama/ollama:0.40.2-rocm
container_name: ollama
restart: unless-stopped
devices:
- "/dev/kfd:/dev/kfd"
- "/dev/dri:/dev/dri"
ports:
- "127.0.0.1:11434:11434"
volumes:
- ollama:/root/.ollama
volumes:
ollama:Confirm the GPU actually landed
A container started with a GPU flag and a model running on the CPU is the normal failure. Nothing in the API response tells you. Three checks, cheapest first:
docker logs ollama
docker exec -it ollama ollama ps
curl http://localhost:11434/api/psThe startup log lists the compute libraries Ollama found. ollama ps has a PROCESSOR column that should read 100% GPU for a model that fits. A CPU/GPU split means only part of it loaded, which is a capacity problem covered in Ollama not using GPU and the VRAM sizing guide. The /api/ps response carries size_vram in bytes, the field to scrape if you graph this. Ollama’s troubleshooting page holds the container-specific fixes: reload nvidia_uvm if the toolkit test passes but Ollama sees nothing, and if the GPU works at start and then drops to CPU later, switch Docker to the cgroupfs cgroup driver in /etc/docker/daemon.json.
A Compose file that survives reboots
For anything left running, a Compose file records the flags and restarts with the host. The reservation syntax comes from Docker’s Compose GPU docs, where capabilities is mandatory and count and device_ids are mutually exclusive:
services:
ollama:
image: ollama/ollama:0.40.2
container_name: ollama
restart: unless-stopped
ports:
- "127.0.0.1:11434:11434"
volumes:
- ollama:/root/.ollama
environment:
OLLAMA_KEEP_ALIVE: "30m"
OLLAMA_MAX_LOADED_MODELS: "1"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
volumes:
ollama:Start it with docker compose up -d and pull a model with docker compose exec ollama ollama pull llama3.2. The environment block changes two defaults from the Ollama FAQ. Models unload after 5 minutes idle, which means a cold reload and a slow first token after any lull. A longer keep-alive trades idle VRAM for that. The loaded-model cap defaults to 3 per GPU, and pinning it to 1 stops a second model from evicting the one you care about on a single card. OLLAMA_NUM_PARALLEL stays at its default of 1. The Compose spec also accepts gpus: all on the service as a shorter alternative to the deploy block. The AMD version is in the ROCm section above.
Build your own Ollama Dockerfile
Extend the published image with FROM rather than building Ollama from source; the upstream Dockerfile compiles CPU, CUDA 12, CUDA 13, ROCm and Vulkan backends in a long multi-stage build. The Ollama FAQ does exactly that for a proxy with a self-signed certificate: FROM ollama/ollama, COPY the certificate to /usr/local/share/ca-certificates/my-ca.crt, then RUN update-ca-certificates.
The more common request is baking a model into the image. ollama pull needs a running server, and there is no single build-time command for it; issue #5385 has asked for one since June 2024. The workaround starts the server in the background for the length of one RUN:
FROM ollama/ollama:0.40.2
RUN ollama serve & sleep 5 && ollama pull llama3.2 && kill $!Raise the sleep on a slow builder. With no VOLUME in the base image, the model stays in the layer and the image grows by its size, about 2.0 GB for llama3.2. Mounting a new, empty named volume at /root/.ollama copies the baked files into it, per Docker’s volume docs, but a volume that already holds data hides them.
Port 11434 is not a login screen
The Compose file above binds the port to 127.0.0.1 on purpose. The stock docker run line publishes on every host interface, which Docker’s port-publishing docs describe as insecure by default. Inside the container the image listens on 0.0.0.0, the process runs as root, and Ollama has no authentication. Wiz’s CVE-2024-37032 disclosure put those together: a path traversal in /api/pull gave remote code execution as root on exposed Docker deployments, was fixed in 0.1.34, and their scan found over 1,000 exposed instances on the public internet, a large number of them on vulnerable versions. The serving layer keeps producing entries like it, which the AI Alert CVE roundup tracks. Bind to localhost, put a reverse proxy with authentication in front for anything remote (the FAQ has an nginx example), and pin the image tag so an upgrade is a decision rather than a surprise on the next docker pull.
FAQ
how do i update the ollama docker image without losing models?
Pull the new tag, remove the old container and start a fresh one on the same named volume: docker pull ollama/ollama:0.40.2, docker rm -f ollama, then your original docker run line with the new tag. Docker keeps a volume’s data after its container is removed, and curl http://localhost:11434/api/version confirms which version is now running.
where does the ollama docker container store models?
Inside the container, models live under /root/.ollama, the path the official run command mounts. On the host that is the named volume ollama, and docker volume inspect ollama prints its Mountpoint, which defaults to a directory under /var/lib/docker/volumes/. Keep that disk roomy, since a single large model can run to tens of gigabytes.
can i run ollama docker with an amd gpu on windows?
Not through Docker Desktop. Its GPU support covers only NVIDIA cards on the WSL 2 backend, so the rocm image cannot reach a Radeon there. Install native Ollama for Windows instead. Its ROCm v7 path supports the Radeon RX 7900 XTX, 7900 XT, 7900 GRE, 7800 XT, 7700 XT, 7600 XT and 7600, plus Radeon PRO W7500 through W7900.
does the ollama docker image run on arm64?
Yes. The latest and numbered release tags are published for linux/arm64 as well as linux/amd64, and the arm64 build of 0.40.2 is a 2.81 GB compressed download. On NVIDIA Jetson boards, Ollama’s Docker docs say to set JETSON_JETPACK to 5 or 6 to match the installed JetPack.
