# localLLMrig full

> Which hardware do you need to run your LLM locally?

Estimate, not a measurement. benchmarkSourceUrl null. Prices: sourceUrl null, as of 2026-10-14.

## Formula

Weight GB ≈ paramsB × bytes. Q4 0.56, Q5 0.69, Q8 1.07, FP16 2.
KV cache is FP16 (2 bytes per element). Overhead 0.8 GB.
Discrete GPU reserve 1.2 GB per card. System RAM reserve 8 GB when offloading.
Utilisation (assumption): NVIDIA 0.55, Apple 0.48, AMD 0.4, mixed 0.5.
Multi-GPU bandwidth factor 0.8. CPU bandwidth utilisation 0.45. Per-token overhead 0.0025 s.
tok/s ≈ 1 / (streamed GB / effective bandwidth + overhead). This is not a benchmark.

## Models at 8192 context

- Llama 3.2 3B (llama-3.2-3b): 3.21B paramsEstimated, Q4 3.54 GB, Q5 3.95 GB, Q8 5.17 GB, FP16 8.16 GB at 8192 tokens. Estimate, memory sourceUrl null. https://huggingface.co/unsloth/Llama-3.2-3B-Instruct/raw/main/config.json.
- Llama 3.1 8B (llama-3.1-8b): 8.03B paramsEstimated, Q4 6.37 GB, Q5 7.41 GB, Q8 10.47 GB, FP16 17.93 GB at 8192 tokens. Estimate, memory sourceUrl null. https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct/raw/main/config.json.
- Mistral 7B v0.3 (mistral-7b-v03): 7.25B paramsEstimated, Q4 5.93 GB, Q5 6.88 GB, Q8 9.63 GB, FP16 16.37 GB at 8192 tokens. Estimate, memory sourceUrl null. architecture sourceUrl null.
- Gemma 2 9B (gemma-2-9b): 9.24B paramsEstimated, Q4 8.09 GB, Q5 9.29 GB, Q8 12.8 GB, FP16 21.39 GB at 8192 tokens. Estimate, memory sourceUrl null. architecture sourceUrl null.
- Qwen2.5 14B (qwen2.5-14b): 14.7B paramsEstimated, Q4 10.64 GB, Q5 12.55 GB, Q8 18.14 GB, FP16 31.81 GB at 8192 tokens. Estimate, memory sourceUrl null. https://huggingface.co/Qwen/Qwen2.5-14B/raw/main/config.json.
- Phi-4 14B (phi-4): 14.7B paramsEstimated, Q4 10.71 GB, Q5 12.62 GB, Q8 18.21 GB, FP16 31.88 GB at 8192 tokens. Estimate, memory sourceUrl null. architecture sourceUrl null.
- Gemma 2 27B (gemma-2-27b): 27.2B paramsEstimated, Q4 18.35 GB, Q5 21.88 GB, Q8 32.22 GB, FP16 57.52 GB at 8192 tokens. Estimate, memory sourceUrl null. architecture sourceUrl null.
- Qwen2.5 32B (qwen2.5-32b): 32.8B paramsEstimated, Q4 21.32 GB, Q5 25.58 GB, Q8 38.04 GB, FP16 68.55 GB at 8192 tokens. Estimate, memory sourceUrl null. https://huggingface.co/Qwen/Qwen2.5-32B/raw/main/config.json.
- Llama 3.3 70B (llama-3.3-70b): 70.6B paramsEstimated, Q4 43.02 GB, Q5 52.2 GB, Q8 79.03 GB, FP16 144.68 GB at 8192 tokens. Estimate, memory sourceUrl null. https://huggingface.co/unsloth/Llama-3.3-70B-Instruct/raw/main/config.json.
- Qwen2.5 72B (qwen2.5-72b): 72.7B paramsEstimated, Q4 44.2 GB, Q5 53.65 GB, Q8 81.27 GB, FP16 148.88 GB at 8192 tokens. Estimate, memory sourceUrl null. https://huggingface.co/Qwen/Qwen2.5-72B/raw/main/config.json.
- DeepSeek-R1-Distill 70B (deepseek-r1-distill-70b): 70.6B paramsEstimated, Q4 43.02 GB, Q5 52.2 GB, Q8 79.03 GB, FP16 144.68 GB at 8192 tokens. Estimate, memory sourceUrl null. architecture sourceUrl null.

## Hardware

- GeForce RTX 3060 12GB (card-3060-12): 12 GB vram, 360 GB/s, EUR 280, price sourceUrl null, estimated, as of 2026-10-14. recommend=false. bandwidth sourceUrl null.
- GeForce RTX 4060 Ti 16GB (card-4060ti-16): 16 GB vram, 288 GB/s, EUR 460, price sourceUrl null, estimated, as of 2026-10-14. recommend=false. bandwidth sourceUrl null.
- GeForce RTX 3090 24GB (card-3090): 24 GB vram, 936 GB/s, EUR 720, price sourceUrl null, estimated, as of 2026-10-14. recommend=false. bandwidth sourceUrl null.
- GeForce RTX 4090 24GB (card-4090): 24 GB vram, 1008 GB/s, EUR 1850, price sourceUrl null, estimated, as of 2026-10-14. recommend=false. bandwidth sourceUrl null.
- GeForce RTX 5090 32GB (card-5090): 32 GB vram, 1792 GB/s, EUR 2790, price sourceUrl null, estimated, as of 2026-10-14. recommend=false. https://www.nvidia.com/en-eu/geforce/graphics-cards/50-series/.
- Radeon RX 7900 XTX 24GB (card-7900xtx): 24 GB vram, 960 GB/s, EUR 920, price sourceUrl null, estimated, as of 2026-10-14. recommend=false. bandwidth sourceUrl null.
- Budget PC with RTX 3060 12GB (build-3060): 12 GB vram, 360 GB/s, EUR 890, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- Mid-range PC with RTX 4060 Ti 16GB (build-4060ti): 16 GB vram, 288 GB/s, EUR 1490, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- Used PC with RTX 3090 24GB (build-3090): 24 GB vram, 936 GB/s, EUR 1280, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- PC with Radeon RX 7900 XTX 24GB (build-7900xtx): 24 GB vram, 960 GB/s, EUR 1790, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- PC with RTX 4090 24GB (build-4090): 24 GB vram, 1008 GB/s, EUR 2890, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- PC with RTX 5090 32GB (build-5090): 32 GB vram, 1792 GB/s, EUR 4490, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://www.nvidia.com/en-eu/geforce/graphics-cards/50-series/.
- Dual RTX 3090, 48 GB (build-dual-3090): 48 GB vram, 1872 GB/s, EUR 1980, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- Dual RTX 4090, 48 GB (build-dual-4090): 48 GB vram, 2016 GB/s, EUR 5490, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- Mac mini M4 16GB (mac-mini-m4-16): 16 GB unified, 120 GB/s, EUR 549, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- Mac mini M4 Pro 64GB (mac-mini-m4-pro-64): 64 GB unified, 273 GB/s, EUR 1890, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. bandwidth sourceUrl null.
- Mac Studio M4 Max 36GB (mac-studio-m4-max-36): 36 GB unified, 410 GB/s, EUR 1890, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://support.apple.com/en-us/122211.
- Mac Studio M4 Max 128GB (mac-studio-m4-max-128): 128 GB unified, 546 GB/s, EUR 3290, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://support.apple.com/en-us/122211.
- Mac mini M6 16GB (mac-mini-m6-16): 16 GB unified, 153 GB/s, EUR 799, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://www.apple.com/mac-mini/specs/.
- Mac mini M6 32GB (mac-mini-m6-32): 32 GB unified, 170 GB/s, EUR 1249, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://www.apple.com/mac-mini/specs/.
- Mac mini M5 Pro 64GB (mac-mini-m5-pro-64): 64 GB unified, 307 GB/s, EUR 2790, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://www.apple.com/mac-mini/specs/.
- Mac Studio M5 Max 36GB (mac-studio-m5-max-36): 36 GB unified, 460 GB/s, EUR 2590, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://www.apple.com/mac-studio/specs/.
- Mac Studio M5 Max 128GB (mac-studio-m5-max-128): 128 GB unified, 614 GB/s, EUR 5190, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://www.apple.com/mac-studio/specs/.
- Mac Studio M5 Ultra 96GB (mac-studio-m5-ultra-96): 96 GB unified, 1200 GB/s, EUR 6490, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://www.apple.com/mac-studio/specs/.
- Mac Studio M5 Ultra 256GB (mac-studio-m5-ultra-256): 256 GB unified, 1200 GB/s, EUR 10490, price sourceUrl null, estimated, as of 2026-10-14. recommend=true. https://www.apple.com/mac-studio/specs/.

## Example recommendations

Each line is an estimate. benchmarkSourceUrl null. price sourceUrl null. Do not treat tokens/s as a measurement.

### Llama 3.3 70B Q4 under EUR 2000
Llama 3.3 70B, Q4, context 8192, need 43.02 GB (estimate, sourceUrl null).
- fastest: Dual RTX 3090, 48 GB (build-dual-3090), full, 18.6 tok/s estimate, EUR 1980, withinBudget=true, benchmarkSourceUrl null, priceSourceUrl null.
- platform: Mac mini M4 Pro 64GB (mac-mini-m4-pro-64), full, 3.08 tok/s estimate, EUR 1890, withinBudget=true, benchmarkSourceUrl null, priceSourceUrl null.
- step-up: Dual RTX 4090, 48 GB (build-dual-4090), full, 19.96 tok/s estimate, EUR 5490, withinBudget=false, benchmarkSourceUrl null, priceSourceUrl null.

### Llama 3.1 8B Q4 under EUR 1000
Llama 3.1 8B, Q4, context 8192, need 6.37 GB (estimate, sourceUrl null).
- fastest: Budget PC with RTX 3060 12GB (build-3060), full, 32.64 tok/s estimate, EUR 890, withinBudget=true, benchmarkSourceUrl null, priceSourceUrl null.
- platform: Mac mini M6 16GB (mac-mini-m6-16), full, 12.76 tok/s estimate, EUR 799, withinBudget=true, benchmarkSourceUrl null, priceSourceUrl null.
- step-up: PC with RTX 5090 32GB (build-5090), full, 122.67 tok/s estimate, EUR 4490, withinBudget=false, benchmarkSourceUrl null, priceSourceUrl null.

### Auto model Q4 under EUR 1500
Qwen2.5 32B, Q4, context 8192, need 21.32 GB (estimate, sourceUrl null).
- fastest: Used PC with RTX 3090 24GB (build-3090), full, 23.61 tok/s estimate, EUR 1280, withinBudget=true, benchmarkSourceUrl null, priceSourceUrl null.
- platform: Mac mini M6 32GB (mac-mini-m6-32), full, 3.94 tok/s estimate, EUR 1249, withinBudget=true, benchmarkSourceUrl null, priceSourceUrl null.
- step-up: PC with RTX 5090 32GB (build-5090), full, 42.89 tok/s estimate, EUR 4490, withinBudget=false, benchmarkSourceUrl null, priceSourceUrl null.

## Guides

English only, at https://localllmrig.com/guides. Each guide uses catalog fields or the formula above. benchmarkSourceUrl stays null.
- Best GPU for local LLMs in 2026: https://localllmrig.com/guides/best-gpu-for-local-llms-2026
- How much VRAM does a 70B model need?: https://localllmrig.com/guides/how-much-vram-for-70b
- Mac vs PC for a local LLM: https://localllmrig.com/guides/mac-vs-pc-for-local-llms
- Used RTX 3090 buying guide (EU): https://localllmrig.com/guides/used-rtx-3090-buying-guide-eu
- Running DeepSeek and Qwen locally: https://localllmrig.com/guides/running-deepseek-and-qwen-locally
- Quantization explained for local LLMs: https://localllmrig.com/guides/quantization-explained
- The RTX 5090 is (reportedly) dead: best 32GB alternatives for local LLMs: https://localllmrig.com/guides/rtx-5090-reportedly-dead-32gb-alternatives
- Is RTX Spark worth it for local LLMs vs Mac Studio vs Strix Halo?: https://localllmrig.com/guides/rtx-spark-vs-mac-studio-vs-strix-halo
- Run Nemotron 3.5 Bolt 74B on a 24GB GPU: https://localllmrig.com/guides/nemotron-3-5-bolt-74b-upcoming
- Run a local LLM on Windows in 5 minutes (Llama Windows): https://localllmrig.com/guides/llama-windows-local-llm
- Best local LLM rig under $500 / $1000 / $3000: https://localllmrig.com/guides/best-local-llm-rig-under-500-1000-3000