Nemotron 3.5 Lightning: Specs, VRAM, and How to Run It
On August 11, 2026, Nvidia released Nemotron 3.5 Lightning—a new member of the Nemotron 3 open-source model family, positioned as a 'lightweight mainstay for long-running Agent workflows'. At the same time, the model routing library NeMo Switchyard was also open-sourced.
This article does not repeat the press release. All parameters below have been verified one by one from the Hugging Face model card and Nvidia official blog, focusing on answering three questions: What it is, whether your card can run it, and how to try it fastest.
Key Parameters Overview (All from Official Model Card)
| Item | Value |
|---|---|
| Total Parameters / Active Parameters | 30B / 3B (MoE) |
| Architecture | Mamba-2 + MoE + Attention hybrid |
| Context Length | Up to 1M tokens |
| Official Quantization | NVFP4 (4-bit, also BF16 version) |
| Single Card Deployment | 1× DGX Spark (GB10) or 1× H100 |
| Supported Hardware | Blackwell (DGX Spark, GB200, GeForce RTX 5090), Hopper (H100/H200), Ampere (via W4A16) |
| License | OpenMDW-1.1, commercial use allowed |
| Languages | English + Spanish/French/German/Italian/Japanese (Chinese not in official support list) |
| Recommended Sampling Parameters | temperature 1.0, top_p 0.95 |
| Data Cutoff | Pre-training 2025-09, post-training 2026-05 |
Two points that are easily misled by the press release, correction first:
- The official hardware list does not include RTX 40 series (Ada architecture). The list only mentions Blackwell, Hopper, Ampere (W4A16 path). RTX 4090 users should not rush just because they see 'RTX can run', wait for community testing or GGUF conversion.
- Its strength is the throughput and VRAM advantages from low activation volume, not comprehensive benchmark dominance—its own HLE score is only 10.47, positioned as 'frequent cheap labor in Agent systems', complex planning still requires larger models.
How Much Accuracy Does NVFP4 Quantization Lose? The Answer is Almost None
Official BF16 vs NVFP4 comparison under the same evaluation framework (NeMo Gym / NeMo Evaluator), here are a few key rows:
| Benchmark | BF16 | NVFP4 |
|---|---|---|
| MMLU Pro | 81.94 | 81.62 |
| GPQA Diamond | 75.44 | 75.57 |
| SWE-bench Verified | 51.56 | 52.80 |
| Terminal-Bench 2.1 | 24.58 | 23.46 |
| IFBench (loose) | 71.88 | 72.88 |
On SWE-bench Verified, the quantized version is actually higher (52.80 vs 51.56). The practical meaning of these numbers is: just use the official NVFP4 checkpoint, no need to worry about accuracy loss.
How Much VRAM is Needed?
Official VRAM numbers are not given, but weight volume can be calculated: 30B parameters × 4 bit ≈ 15GB weight, plus KV cache and runtime overhead, 24-32GB VRAM tier is a reasonable starting point—this falls exactly within the official support range for RTX 5090 (32GB). Running full 1M context is another matter, KV cache expands linearly with context, and the official vLLM recipe defaults to --kv-cache-dtype fp8.
For precise estimation based on your own card and context length, you can use our LLM VRAM Calculator.
Three Ways to Get Started, from Lowest to Highest Cost
1. Zero-Cost Online Trial (No Installation)
- build.nvidia.com official playground to chat directly
- OpenRouter has a free tier (
nvidia/nemotron-3.5-lightning:free), which can be connected directly via API for testing
2. vLLM Local Deployment (Official Recipe)
The DGX Spark recipe given in the model card, key points are marlin MoE backend + fp8 KV cache + DSpark speculative decoding:
export MODEL_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4
export DSPARK_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark
vllm serve --model $MODEL_CKPT \
--moe-backend marlin \
--kv-cache-dtype fp8 \
--enable-prefix-caching \
--speculative_config.num_speculative_tokens 3
(vLLM requires a relatively new version, officially marked as vllm/vllm-openai:v0.27.1.)
3. NIM Microservice / On Cloud
The enterprise path is via NVIDIA NIM, and HF, ModelScope also have weight mirrors.
NeMo Switchyard Released Alongside is Worth a Look
Switchyard is an open-source model routing library: it automatically routes each request in Agent workflows to 'the cheapest model that is sufficient'. The numbers from collaborators are quite convincing—LangChain in 145 multi-turn Deep Agents tasks only sent 7% of calls to the flagship model, reducing cost by 74% (with 6% accuracy loss); Ramp cut 58% cost and 33% runtime on their own SWE-Bench.
This is a complementary strategy to Lightning: the router assigns high-frequency simple tasks to the cheap model with 3B activation, and only uses the flagship for difficult problems. If you are building a multi-Agent system, this 'system-level cost-saving' approach is worth evaluating more than just switching models. Model unit prices can be compared horizontally in LLM API Pricing Comparison.
Frequently Asked Questions (FAQ)
How Much VRAM Does Nemotron 3.5 Lightning Need?
NVFP4 weights are about 15GB (30B × 4 bit), plus KV cache, 24GB or more VRAM is recommended to start. Official single card reference is 1× DGX Spark (GB10) or 1× H100; among consumer cards, RTX 5090 (32GB) is in the official support list.
Can it be used commercially?
Yes. The license is OpenMDW-1.1, and the model card explicitly states 'ready for commercial use'.
Can RTX 4090 run it?
The official support list only includes Blackwell, Hopper, and Ampere (W4A16 path), Ada architecture (RTX 40 series) is not listed. It is recommended to wait for community test results before deciding.
Does it support Chinese?
The official language list is English, Spanish, French, German, Italian, Japanese; Chinese is not included. It supports 43 programming languages.
Can thinking mode be turned off?
Yes. Reasoning control is exposed via chat-template kwargs: thinking is enabled by default, but can be switched to direct response mode.