Best Ollama Models for 8GB VRAM: What Fits

An 8 GB card is the most common entry point into local models, and it is also the tier where the gap between “fits” and “nearly fits” does the most damage. A model that sits entirely in VRAM and the same model with a handful of layers pushed onto the CPU differ by roughly an order of magnitude in generation speed. At 24 GB you can be careless about this. At 8 GB you cannot.

So the question “which model should I run on 8 GB” is really a memory arithmetic question, and it has a clean answer once you stop treating 8 GB as a budget you get to spend.

You do not have 8 GB

The number on the box is installed memory, not available memory. Before Ollama loads a single layer, some of it is already gone:

• The display output. A desktop session with a compositor and a hardware-accelerated browser routinely holds 0.7 to 1.5 GB. On Windows, the desktop window manager alone reserves a few hundred megabytes.
• Driver and context overhead. Allocating a CUDA or ROCm context costs memory before any weights arrive.
• Compute buffers. Ollama reserves working space for the forward pass, separate from weights and separate from the KV cache.

The practical figure to plan against:

Machine	Realistic free VRAM on an 8 GB card
Headless Linux, no display attached	7.3 to 7.6 GB
Linux desktop, browser closed	6.8 to 7.2 GB
Windows or Linux desktop, browser open	6.0 to 6.8 GB

Check yours rather than assuming, because this single number decides everything downstream:

nvidia-smi --query-gpu=memory.total,memory.used,memory.free --format=csv

Plan against roughly 6.5 GB usable on a machine you also use as a desktop. That is the real budget.

The arithmetic, briefly

At the Q4_K_M quantisation most of the Ollama library ships by default, weights cost about 0.6 GB per billion parameters. On top of that sits the KV cache, which grows linearly with context length, plus 0.5 to 1 GB of buffers.

The KV cache cost per token is not universal. It depends on layer count, key/value head count, and head dimension, so a model using grouped-query attention costs a small fraction of what an older multi-head model of the same parameter count costs. This is why two 8B models can differ substantially in how much context they will hold on the same card. The VRAM calculator works the sum for a specific model, quantisation, and window, and how much VRAM you need to run local LLMs with Ollama explains where each term comes from.

What fits in 8 GB

Weight estimates below use the 0.6 GB per billion rule at Q4_K_M, against a 6.5 GB working budget.

Parameter class	Q4_K_M weights	Left for cache and buffers	Verdict on 8 GB
1-2B	0.6 to 1.2 GB	5.3 to 5.9 GB	Very long context, very fast
3-4B	1.8 to 2.4 GB	4.1 to 4.7 GB	Comfortable, long context
7-8B	4.2 to 4.8 GB	1.7 to 2.3 GB	Fits, moderate context
9B	~5.4 GB	~1.1 GB	Tight, short context only
12-14B	7.2 to 8.4 GB	None	Does not fit at Q4
27-32B	16 to 19 GB	None	Not at any usable quantisation

The 7-8B row is the one people care about, and it is genuinely fine. An 8B model at Q4_K_M leaves around 2 GB for cache, which on a modern grouped-query-attention model is enough for a context window in the 8K to 16K range. That covers ordinary chat, code assistance on a file at a time, and most tool-calling work.

The 12-14B row is where the disappointment happens. A 14B model at Q4_K_M does not fit in 8 GB, and it will not tell you so. Ollama will load what it can and run the remainder on the CPU, which is the failure mode described in Ollama not using GPU. If you want a 14B model on the GPU, that is a 16 GB card, not an 8 GB one with clever settings.

The tradeoff you actually control: parameters against context

With a fixed 6.5 GB, every gigabyte of context is a gigabyte not spent on parameters. Three sensible ways to spend it:

• Maximum capability, short window. An 8B model at Q4_K_M with a 4K to 8K window. Best single-turn answer quality available at this tier.
• Balanced. A 3-4B model with a 32K window or beyond. Noticeably weaker per token, but it can hold a whole document, and it generates two to three times faster because it reads fewer bytes per token.
• Long-context work. A 1-3B model with a very large window, for summarisation and extraction over long inputs where retrieval and structure matter more than reasoning depth.

Ollama’s default context window scales with available VRAM rather than being fixed. The context length documentation sets it at 4K below 24 GiB, 32K from 24 to 48 GiB, and 256K above that. Every 8 GB card falls in the first bucket, so the default you get is 4K, no matter that the model advertises 128K. That is worth knowing in both directions: a model appearing to forget the top of a long document is usually this, and raising num_ctx to the advertised maximum is one of the most reliable ways to force partial CPU offload, because the cache grows until it no longer fits and nothing warns you.

Two settings buy back context cheaply:

OLLAMA_KV_CACHE_TYPE=q8_0
OLLAMA_FLASH_ATTENTION=1

Quantising the KV cache to q8_0 roughly halves its memory cost at negligible quality cost. Flash attention reduces attention working memory as the window grows, and Ollama enables it automatically where the backend supports it, so setting it explicitly mainly serves to confirm it is on.

Do not reach for lower quantisation first

The instinct at 8 GB is to run a bigger model at Q3 or Q2 rather than a smaller one at Q4. This is usually the wrong trade.

Below roughly 4 bits per weight, degradation stops being subtle. It shows up first in exactly the places local models are most useful: structured output, JSON that has to parse, tool-call arguments, and long-chain reasoning. A 14B model at Q3 that produces malformed JSON is worth less than an 8B model at Q4_K_M that does not.

The ordering that costs the least quality:

1. Reduce the context window to what you need.
2. Quantise the KV cache to q8_0.
3. Step down a parameter class, for example 8B to 4B.
4. Only then step below Q4_K_M.

Speed at this tier

Once a model fits, generation speed is set by memory bandwidth, not by the card’s compute. Bandwidth varies more than twofold across cards that all carry 8 GB, using published reference specifications cross-referenced against the TechPowerUp database:

Card	Bus width	Bandwidth	Ceiling for an 8B at Q4
RTX 3050 8GB	128-bit GDDR6	~224 GB/s	~47 tok/s
RTX 4060 8GB	128-bit GDDR6	~272 GB/s	~57 tok/s
RTX 4060 Ti 8GB	128-bit GDDR6	~288 GB/s	~60 tok/s
RX 7600 8GB	128-bit GDDR6	~288 GB/s	~60 tok/s
RTX 3070 8GB	256-bit GDDR6	~448 GB/s	~93 tok/s
Arc A750 8GB	256-bit GDDR6	~512 GB/s	~107 tok/s

Ceilings are bandwidth divided by the 4.8 GB of weights an 8B model at Q4_K_M reads per token. Real rates land near 60 to 80 percent of that, which the method in Ollama tokens per second works through.

Every one of those figures is comfortably faster than reading speed. This is the reassuring part of the 8 GB tier: capacity is the constraint, speed is not. A 7-8B model that fits will feel instant on any of these cards.

Where the 8 GB tier runs out

It runs out at capability, not speed. The 27-32B class is where local output starts to feel qualitatively different from the small models, and no amount of tuning reaches it from 8 GB. If that is the target, the answer is a different card, and best GPU for Ollama compares the tiers by capacity and bandwidth together.

Two alternative routes are worth knowing before spending on a discrete upgrade. Apple Silicon addresses system memory directly, so a Mac reaches large-model capacity without a card at all, at the bandwidth its chip provides; the sizing is covered in Ollama on Apple Silicon. And if the workload is offline rather than interactive, Ollama without a GPU sets out what CPU-only inference can and cannot do, which occasionally makes a purchase unnecessary.

Checklist for an 8 GB card

1. Measure free VRAM with nvidia-smi, with your normal desktop running.
2. Subtract 1 GB for buffers. That is the model plus cache budget.
3. Pick the largest parameter class fitting that budget at Q4_K_M, usually 7-8B.
4. Set num_ctx explicitly rather than accepting a default or a maximum.
5. Confirm ollama ps reports 100% GPU.
6. If it does not, cut context before cutting quantisation.

Step 5 is the one that gets skipped, and it is the one that determines whether the machine is fast or ten times slower than it should be.