Best GPU for Local LLMs in 2026

The short answer: three GPUs worth buying (2026)

These are three capacity candidates, not unconditional buying recommendations. “Best value” is a decision to validate with your local quote; cost per million tokens and performance per dollar are Not yet verified. No speed ranking is inferred from gaming specifications or from a small-model benchmark.

Use the memory assumptions before treating a weight-only fit as a runnable model. A real model name and verified artifact fit are pending for each candidate; nominal parameter labels do not identify a downloadable checkpoint.

How much VRAM you actually need

The rows are nominal parameter-count scenarios, not exact counts of named models. Every weight cell is EST. binary GiB calculated from the official sample ratio; no cell is a peak GPU allocation. Capacity comparisons elsewhere retain decimal GB, with exact unit conversion.

EST. weight GiB = nominal parameters (billions) × sample bits per weight × (10⁹ ÷ 8 ÷ 1024³). The conversion is calculated exactly, rather than using a rounded decimal constant. EST. weight GB = nominal parameters (billions) × sample bits per weight ÷ 8.

All ratios, including F16 and Q8_0, come from llama.cpp's Llama-3.1-8B example and retain the published precision. Q8_0 is not raw INT8. BF16 has the same nominal storage width as FP16, but an actual mixed-format artifact still needs inspection. Applying sample ratios to other scales is an explicit extrapolation: tensor mixtures, embeddings, vocabulary and conversion options can change the result. Actual GGUF size for each row is Not yet verified; no universal percentage error bound is established by this source.

These estimates exclude KV cache, runtime buffers, activations, vision components and allocator overhead. A nominal parameter label alone cannot explain every difference from a published file size. Compare the exact artifact and byte count instead of forcing the nominal-size estimate to match it.

For a conventional attention cache: EST. KV GB = 2 × layers × KV heads × head dimension × cached tokens × concurrent sequences × cache bits ÷ 8 ÷ 10⁹. Architecture and cache configuration must be known; sliding-window or hybrid models can require a different formula.

KV cache from three readable official configurations

Head dimension = hidden_size ÷ num_attention_heads. The following examples use the configured context length and a single sequence with a full-context cache. Configs specify BF16 model weights; F16 here is an explicit cache-payload scenario, not a claim about the runtime's default cache precision.

The raw 8-bit column is payload-only arithmetic. It is not the actual Q8_0 cache size: block scales, supported cache formats and allocation behavior need runtime verification. Likewise, a raw 4-bit payload is not proof of actual Q4 cache memory. No fixed extra memory allowance or long-context performance guarantee is asserted. All other models retain Not yet verified KV values.

VRAM calculator — planned · Choose a model briefing · Mistral Large 4 VRAM notes

VRAM tiers: what 8, 12, 16, 24, 32, 48, 96GB each buy you

“Headroom” below is an EST. screening rule: reserve 20% of nominal capacity after allocating the estimated weights. It is an editorial margin, not a verified KV allowance or a promise of comfortable speed. “Tight” fits the estimated weights but exceeds that screening limit; “cannot hold” means the weights alone exceed capacity.

For every tier, the largest model that will run comfortably at your target context is Not yet verified. Confirm the exact artifact and peak allocation before buying. Lower quantization may introduce quality tradeoffs, and a model spilling into CPU memory has a different latency profile.

Picks by budget

These dollar bands are user-selected budget scenarios, not verified market-price brackets. Each card below is a candidate to obtain a quote for; inclusion does not mean it is currently sold inside that band. Current prices and price-per-GB comparisons are pending.

Compare delivered total including tax, a suitable power supply, cooling and any system upgrade. A card with more memory is not automatically cheaper to own or faster for your workload.

Used RTX 3090 vs new RTX 5060 Ti / 5070

A used card becomes a defensible choice only if its delivered cost per GB, verified condition and workload support beat your alternatives after accounting for operating costs. Estimated electricity cost is Not yet verified: rated board power cannot replace measured whole-system draw or a local tariff.

• Ask about transferability of warranty and the return window; do not assume either exists.
• Inspect connectors, cooling, fan noise and thermal behavior under a sustained load.
• Ask for prior usage and repair history. Mining history or remaining life cannot be inferred from a listing or model name.
• Confirm case clearance, power connectors and a power supply matched to the full system.

NVIDIA's runtime support matrix distinguishes FP8 weight execution from FP8 cache support. Ampere is not listed for per-tensor FP8 weights; Blackwell is, but an FP8 checkpoint still needs a compatible kernel and model. File loading or conversion is not proof of native FP8 acceleration in Ollama, LM Studio or llama.cpp.

Get the missing tariff and usage inputs in the cost comparison before declaring the used card an economic winner.

24GB vs 32GB: what the extra money buys

The nominal upgrade adds EST. 8.00 GB of memory, an EST. 33.33% capacity increase. That is capacity arithmetic, not a speed improvement. The actual price premium is Price check pending.

1. For the smaller case, retain the same model and format, then check whether your target context and concurrency fit the additional headroom.
2. For the larger case, validate quality at the lower-bit format. Offload may make loading possible, but RAM, transport and latency can change the decision.
3. If you need multiple concurrent contexts, spend the extra capacity on cache only after calculating the real cache layout.

EST. dollars per added GB = delivered price difference ÷ added capacity. Its numeric value, additional context tokens and offload speed are Not yet verified. Compare ownership costs before accepting a premium.

Inference vs fine-tuning vs production serving

Inference starts with resident weights, then adds caches and runtime allocations. Decode and prompt processing stress different parts of the system; enough VRAM makes a configuration feasible but does not determine its speed.

Fine-tuning adds training activations, gradients and optimizer state. Adapter training can reduce some allocations, but method, precision, sequence length and batch configuration must be specified before assigning a memory requirement.

Production serving adds concurrent caches, scheduling, startup behavior and availability requirements. A single-user fit is not a serving capacity test. Record both time to first token and sustained decoding under representative concurrent load; this guide has no comparable performance record or performance-per-dollar score.

Which GPU for Ollama, LM Studio and llama.cpp

Choose the exact model format and operating system before the runtime. Backend compatibility is necessary, while the model's weights and cache remain the main capacity inputs; defaults and offload policies can change peak allocation.

• Ollama hardware support: verify NVIDIA or AMD support against the current OS-specific list. The Radeon candidate is listed, but a supported backend is not a memory or speed guarantee.
• LM Studio requirements: confirm the current application and runtime support on your machine; use its model-loading controls to observe allocation.
• llama.cpp documentation: select a supported CUDA, HIP, Metal, Vulkan or CPU/GPU hybrid backend. Validate the binary's options for your installed version.

Apple Silicon uses unified system memory. A supported Metal backend can share it with inference, but the OS and other applications also consume it. Do not equate installed unified memory with dedicated VRAM, or transfer this guide's card-power figures to a Mac.

Buy vs rent vs API: where the crossover is

No numeric crossover is asserted. The inputs below are intentionally empty until a dated quote, local tariff and representative workload are supplied; this avoids turning an assumed utilization rate into a buying recommendation.

EST. monthly electricity = measured whole-system watts ÷ 1,000 × active hours × local USD/kWh. Current result: Not yet verified. No local tariff or idle-load record is available.

EST. crossover output tokens/month = fixed monthly ownership cost ÷ (effective API USD per output token − local variable USD per output token). Current result: Not yet verified. If the denominator is not positive, this simplified model has no positive cost crossover.

Below a verified positive threshold, an API can cost less under the specified workload; above it, ownership may be cheaper. Privacy, uptime, model quality and maintenance remain separate constraints. Include the rest of the PC, financing, repairs and idle electricity for a complete ownership comparison; the displayed card-only formula is a partial cost model.

Performance per dollar is also Not yet verified. Divide comparable output tokens per second by a dated delivered hardware price only after fixing model, quantization, context and concurrency; do not substitute vendor AI TOPS.

Setup: driver → runtime → pull a model → measure your own speed

Install the supported vendor driver, confirm your GPU is detected, then install a runtime from its official documentation. Select an available model whose exact quantized artifact fits your system; the following commands are templates, not evidence that a future model is downloadable.

# Replace MODEL_TAG with an available model tag from your chosen model's documentation.
ollama pull MODEL_TAG
ollama show MODEL_TAG
ollama run MODEL_TAG
ollama ps

Official Ollama quickstart · Generate API timing fields · llama.cpp benchmark documentation.

Use a fixed prompt and record cold and warm runs separately. The generate API exposes prompt and output token counts with their durations. Compute output tokens/second as eval_count ÷ eval_duration × 10⁹ when the duration is in nanoseconds; do not combine prompt evaluation and decoding into a single unexplained speed figure.

The Mistral release announcement includes a future weight-release roadmap. Check its current download and runtime availability before attempting local setup; this guide supplies no Mistral Large 4 pull command or consumer-card fit claim.

FAQ

Questions cover the supplied search intents and official runtime documentation. They are not presented as a verified live Google People Also Ask sample.

Methodology & sources

Reviewed 2026-10-10. Initial edition: source-linked capacity shortlist, shared weight arithmetic, conditional budget scenarios and an unfilled cost framework. No hardware was purchased for this guide. There are no site-owned speed, power or benchmark observations.

• Every numeric data field carries a source URL, access date and evidence kind. Official specification data is distinct from our arithmetic and editorial assumptions.
• The weight planning table uses binary GiB; GPU-capacity comparisons use decimal GB. Conversions are exact. Planning scales are nominal inputs; sample ratios are extrapolated, not universal constants. Actual artifact sizes remain pending.
• The screening reserve is EST. 20% of nominal capacity, chosen for this table only. It does not validate a particular KV cache, runtime buffer or comfortable throughput.
• Memory arithmetic, fit classification and capacity differences come from the shared pure functions used throughout this page. Missing architecture or cost inputs return unknown rather than zero.
• Local output-token cost amortizes card price and adds electricity; rental cost normalizes billed hours; effective API cost includes input as well as output charges. Different quality, caching or concurrency prevents a fair price-only comparison.
• Electricity tariff, whole-system draw, idle load and amortization period are pending. No region-independent tariff or rated-power-as-wall-power assumption is used.
• GPU prices, used transaction records, quotes, rental plans and model-specific API costs are pending. Budget labels are planning scenarios, not market quotes.

Consult the model directory and Mistral VRAM research for model-specific uncertainty. Return to Local LLM guides or the planned calculator entry; the calculator is not implemented in this edition.