What LLMs can a RTX 3060 12GB run?
At the recommended quantization with 16K context, the largest model a RTX 3060 12GB can run is Qwen3.5-9B (9.7B, about 7.5 GB); 3 models on the list run comfortably. With 360GB/s of memory bandwidth, the decode ceiling is roughly bandwidth ÷ weight size; measured speeds usually land at 70-85% of that. We have not benchmarked this machine ourselves yet — submissions welcome. Data verified 2026-10-03.
Specs: 12GB · 360GB/s
Model fit list (recommended quant, 16K context)
Comfortable = ≤ 75% of memory; Tight = ≤ 92%; Lower quant = the recommended quant doesn't fit but a smaller one does. Ceiling = 360GB/s ÷ weight size; single-request decoding cannot exceed it.
| Model | Quant | Needs | Verdict | Decode ceiling tok/s |
|---|---|---|---|---|
| Qwen3-4B | Q4_K_M (GGUF) | 6.0 GB | Comfortable | ≤ 158 |
| Qwen3.5-9B | Q4_K_M (GGUF) | 7.5 GB | Comfortable | ≤ 65 |
| Llama 3.1 8B | Q4_K_M (GGUF) | 8.0 GB | Comfortable | ≤ 79 |
| Qwen3-14B | Q3_K_M (GGUF) | 10.5 GB | Lower quant | ≤ 56 |
| gpt-oss-20B | Q2_K (GGUF) | 9.1 GB | Lower quant | ≤ 53 |
| Gemma 4 26B-A4B | Q2_K (GGUF) | 10.2 GB | Lower quant | ≤ 43 |
| Qwen3.8-27B | Q2_K (GGUF) | 10.8 GB | Lower quant | ≤ 41 |
| Nemotron 3.5 Lightning (30B-A3B) | NVFP4 | 16.4 GB | Won't fit | — |
| Mistral Small 3.x 24B | Q4_K_M (GGUF) | 17.6 GB | Won't fit | — |
| GLM-4.7-Flash (30B-A3B) | Q4_K_M (GGUF) | 20.1 GB | Won't fit | — |
| Qwen3-Coder-30B-A3B | Q4_K_M (GGUF) | 20.3 GB | Won't fit | — |
| Gemma 4 31B | Q4_K_M (GGUF) | 20.5 GB | Won't fit | — |
| Qwen3.6-35B-A3B | Q4_K_M (GGUF) | 22.4 GB | Won't fit | — |
| Qwen3-32B | Q4_K_M (GGUF) | 23.7 GB | Won't fit | — |
| Gemma 3 27B | Q4_K_M (GGUF) | 24.6 GB | Won't fit | — |
| Llama 3.3 70B | Q4_K_M (GGUF) | 47.9 GB | Won't fit | — |
| gpt-oss-120B | MXFP4 | 63.5 GB | Won't fit | — |
| Mistral Leanstral 1.5 (119B-A6B MoE) | Q4_K_M (GGUF) | 73.4 GB | Won't fit | — |
| Qwen3.8-Flash-Next (180B MoE) | Q4_K_M (GGUF) | 111 GB | Won't fit | — |
| Qwen3-235B-A22B | Q4_K_M (GGUF) | 147 GB | Won't fit | — |
| DeepSeek V4 Flash (304B MoE) | Q4_K_M (GGUF) | 187 GB | Won't fit | — |
| GLM-4.5 (355B MoE) | Q4_K_M (GGUF) | 224 GB | Won't fit | — |
| DeepSeek V3 / R1 (671B MoE) | Q4_K_M (GGUF) | 413 GB | Won't fit | — |
| GLM-5.3 (753B MoE) | Q4_K_M (GGUF) | 463 GB | Won't fit | — |
| Kimi K2 (1T MoE) | Q4_K_M (GGUF) | 631 GB | Won't fit | — |
| DeepSeek V4 Pro (1.6T MoE) | Q4_K_M (GGUF) | 983 GB | Won't fit | — |
| Kimi K3 (2.8T MoE) | MXFP4 | 1.46 TB | Won't fit | — |
Benchmarked a RTX 3060 12GB yourself?
Send us the model, quant, runtime version, tok/s and peak memory. Verified readings are added to this page with credit. First-hand numbers with a command or log only.
Submit a benchmarkFAQ
What is the largest LLM a RTX 3060 12GB can run?
At the recommended quantization with 16K context, the ceiling is Qwen3.5-9B (about 7.5 GB, 63% of 12GB). The OS and display reserve 0.5-1GB of VRAM, and llama.cpp / vLLM need headroom for CUDA graphs and activation buffers.
Which models run best on a RTX 3060 12GB?
Models under 9GB leave room for long context without closing other apps, e.g. Qwen3.5-9B, Llama 3.1 8B, Qwen3-4B.
How many tokens per second does a RTX 3060 12GB get?
Single-request decoding is bandwidth-bound: ceiling ≈ 360GB/s ÷ weight size. A 15GB 27B 4-bit model tops out around 24 tok/s; measured speeds are usually 70-85% of that, and speculative decoding (a draft model) adds another 1.5-2x.
Are these numbers measured or estimated?
The fit table is a planning estimate (same formula as the calculator). The first-hand benchmark table contains readings we took on this exact machine with real model files; every row links to the source article.