LLM VRAM Calculator — Can I Run It?

How to use

Pick a model from the presets (Kimi K3, Kimi K2, DeepSeek V3/R1, GLM-4.5, Qwen3, gpt-oss, Llama 3.x, Gemma 3, Mistral) or enter a custom parameter count. Choose a quantization — FP16, FP8, MXFP4, or GGUF quants like Q4_K_M — and a context length. The calculator instantly shows how much VRAM you need, broken down into model weights, KV cache, and runtime overhead, plus a 'Will it fit?' table covering common GPUs (RTX 4090, RTX 5090, A100, H100, H200, B200, MI300X) and Apple Silicon Macs with unified memory. Everything runs locally in your browser — nothing is uploaded.

Why use this tool

Every open-weight model release raises the same question: how much VRAM do I need to run this locally? Working it out by hand is error-prone, because the VRAM requirements for a local LLM come from four moving parts — parameter count, quantization bytes-per-weight, KV cache growth with context length, and runtime overhead. This calculator does that math for the latest open-weight models, including trillion-parameter MoE flagships like Kimi K3 and DeepSeek R1, whose MLA-compressed KV caches behave nothing like dense models. Check it before renting GPUs, buying a Mac for local inference, or starting a 1 TB weight download — 'one RTX 4090' and '8× H100' are very different budgets, and quantization is usually the knob that decides which one you are in. Estimates are for planning; real usage varies slightly by runtime (llama.cpp, vLLM, SGLang, MLX).

FAQ

How much VRAM do I need for a local LLM?

It depends far more on quantization than on the model name. A useful rule of thumb at Q4_K_M (~4.8 bits per weight): weights take roughly 0.6 GB per billion parameters, so an 8B model needs about 5 GB and a 70B model about 42 GB — before KV cache and runtime overhead. That puts 7-8B models comfortably on an 8 GB card, 30B-class models on a 24 GB card, and 70B on two 24 GB cards or a 64 GB+ Mac. Pick your exact model and context length above for a real number instead of a rule of thumb.

How much VRAM does quantization actually save?

FP16 stores 2 bytes per parameter. Q8_0 halves that, Q4_K_M is about 4.8 bits (~0.6 bytes) for roughly a 3.3× saving, and MXFP4 averages just over 4 bits (~0.53 bytes). So a model that needs 140 GB of weights at FP16 drops to around 42 GB at Q4_K_M. Quantization applies to weights; you can shrink the KV cache separately by switching it to FP8.

How much VRAM do I need to run Kimi K3?

Kimi K3 is a 2.8-trillion-parameter MoE model shipped in MXFP4. Weights alone are roughly 1.4 TB, so even at its native quantization you need a multi-GPU server — around 8× H200 141GB or more depending on context length. It is far beyond any consumer GPU or Mac; for most people the practical way to use K3 is a hosted API.

Can I run DeepSeek R1 on a single GPU?

Not the full 671B model. At Q4 quantization the weights alone are about 380 GB, requiring roughly 5-6× H100 80GB. Single-GPU users should look at distilled variants (e.g. R1-Distill-Qwen-32B), which fit on a 24 GB card at Q4_K_M.

What is MXFP4 quantization?

MXFP4 is a 4-bit floating-point microscaling format. Weights are stored in blocks that share a scaling factor, averaging just over 4 bits (~0.53 bytes) per parameter. Kimi K3 and OpenAI's gpt-oss models ship natively in MXFP4, which is why their download sizes are about a quarter of FP16.

How does context length affect VRAM usage?

Each token in the context stores keys and values (the KV cache) for every layer, so cache size grows linearly with context length: for a dense 70B model at FP16 it is roughly 40 GB at 128K tokens — enough to double your total VRAM requirement on its own. MLA-based models (DeepSeek, Kimi) compress this dramatically, and the FP8 KV cache option halves it again.

Which GGUF quantization should I choose?

Q4_K_M is the community default — about 4.8 bits per weight with modest quality loss. Use Q5_K_M or Q6_K if you have spare memory, Q8_0 for near-lossless quality, and Q3/Q2 only when nothing else fits — quality degrades noticeably below 4 bits.

Do the VRAM estimates include the KV cache and overhead?

Yes. The total is weights + KV cache at your chosen context length + runtime overhead (buffers, activations, fragmentation — about 8% of weights with a 1.5 GB floor). Many simpler VRAM calculators only count weights, which badly underestimates real usage at long context.

How accurate are these numbers?

They are planning estimates, typically within ~10% of real usage at batch size 1. Actual consumption varies by runtime and settings: vLLM pre-allocates KV blocks, llama.cpp offloads layers, and MLX on Apple Silicon shares unified memory with the OS. Very new models with unpublished architecture details are marked as estimated.

Is my data uploaded anywhere?

No. The calculator is pure client-side JavaScript — model specs are bundled with the page and all math runs in your browser. No sign-up, no upload, no tracking of your inputs.