What LLMs can a RTX 5090 32GB run?
At the recommended quantization with 16K context, the largest model a RTX 5090 32GB can run is Qwen3.6-35B-A3B (36B, about 22.4 GB); 14 models on the list run comfortably. With 1792GB/s of memory bandwidth, the decode ceiling is roughly bandwidth ÷ weight size; measured speeds usually land at 70-85% of that. We have not benchmarked this machine ourselves yet — submissions welcome. Data verified 2026-10-03.
Specs: 32GB · 1792GB/s
Model fit list (recommended quant, 16K context)
Comfortable = ≤ 75% of memory; Tight = ≤ 92%; Lower quant = the recommended quant doesn't fit but a smaller one does. Ceiling = 1792GB/s ÷ weight size; single-request decoding cannot exceed it.
| Model | Quant | Needs | Verdict | Decode ceiling tok/s |
|---|---|---|---|---|
| Qwen3-4B | Q4_K_M (GGUF) | 6.0 GB | Comfortable | ≤ 789 |
| Qwen3.5-9B | Q4_K_M (GGUF) | 7.5 GB | Comfortable | ≤ 325 |
| Llama 3.1 8B | Q4_K_M (GGUF) | 8.0 GB | Comfortable | ≤ 394 |
| Qwen3-14B | Q4_K_M (GGUF) | 12.4 GB | Comfortable | ≤ 213 |
| gpt-oss-20B | MXFP4 | 12.6 GB | Comfortable | ≤ 173 |
| Nemotron 3.5 Lightning (30B-A3B) | NVFP4 | 16.4 GB | Comfortable | ≤ 121 |
| Gemma 4 26B-A4B | Q4_K_M (GGUF) | 16.5 GB | Comfortable | ≤ 122 |
| Qwen3.8-27B | Q4_K_M (GGUF) | 17.3 GB | Comfortable | ≤ 117 |
| Mistral Small 3.x 24B | Q4_K_M (GGUF) | 17.6 GB | Comfortable | ≤ 131 |
| GLM-4.7-Flash (30B-A3B) | Q4_K_M (GGUF) | 20.1 GB | Comfortable | ≤ 101 |
| Qwen3-Coder-30B-A3B | Q4_K_M (GGUF) | 20.3 GB | Comfortable | ≤ 103 |
| Gemma 4 31B | Q4_K_M (GGUF) | 20.5 GB | Comfortable | ≤ 101 |
| Qwen3.6-35B-A3B | Q4_K_M (GGUF) | 22.4 GB | Comfortable | ≤ 88 |
| Qwen3-32B | Q4_K_M (GGUF) | 23.7 GB | Comfortable | ≤ 99 |
| Gemma 3 27B | Q4_K_M (GGUF) | 24.6 GB | Tight | ≤ 117 |
| Llama 3.3 70B | Q4_K_M (GGUF) | 47.9 GB | Won't fit | — |
| gpt-oss-120B | MXFP4 | 63.5 GB | Won't fit | — |
| Mistral Leanstral 1.5 (119B-A6B MoE) | Q4_K_M (GGUF) | 73.4 GB | Won't fit | — |
| Qwen3.8-Flash-Next (180B MoE) | Q4_K_M (GGUF) | 111 GB | Won't fit | — |
| Qwen3-235B-A22B | Q4_K_M (GGUF) | 147 GB | Won't fit | — |
| DeepSeek V4 Flash (304B MoE) | Q4_K_M (GGUF) | 187 GB | Won't fit | — |
| GLM-4.5 (355B MoE) | Q4_K_M (GGUF) | 224 GB | Won't fit | — |
| DeepSeek V3 / R1 (671B MoE) | Q4_K_M (GGUF) | 413 GB | Won't fit | — |
| GLM-5.3 (753B MoE) | Q4_K_M (GGUF) | 463 GB | Won't fit | — |
| Kimi K2 (1T MoE) | Q4_K_M (GGUF) | 631 GB | Won't fit | — |
| DeepSeek V4 Pro (1.6T MoE) | Q4_K_M (GGUF) | 983 GB | Won't fit | — |
| Kimi K3 (2.8T MoE) | MXFP4 | 1.46 TB | Won't fit | — |
Benchmarked a RTX 5090 32GB yourself?
Send us the model, quant, runtime version, tok/s and peak memory. Verified readings are added to this page with credit. First-hand numbers with a command or log only.
Submit a benchmarkFAQ
What is the largest LLM a RTX 5090 32GB can run?
At the recommended quantization with 16K context, the ceiling is Qwen3.6-35B-A3B (about 22.4 GB, 70% of 32GB). The OS and display reserve 0.5-1GB of VRAM, and llama.cpp / vLLM need headroom for CUDA graphs and activation buffers.
Which models run best on a RTX 5090 32GB?
Models under 24GB leave room for long context without closing other apps, e.g. gpt-oss-20B, Nemotron 3.5 Lightning (30B-A3B), Qwen3.6-35B-A3B, Qwen3-Coder-30B-A3B.
How many tokens per second does a RTX 5090 32GB get?
Single-request decoding is bandwidth-bound: ceiling ≈ 1792GB/s ÷ weight size. A 15GB 27B 4-bit model tops out around 119 tok/s; measured speeds are usually 70-85% of that, and speculative decoding (a draft model) adds another 1.5-2x.
Are these numbers measured or estimated?
The fit table is a planning estimate (same formula as the calculator). The first-hand benchmark table contains readings we took on this exact machine with real model files; every row links to the source article.