Your Terminal Just Got a Coworker: Why Local AI Coding Agents Are Eating the Cloud
Your Terminal Just Got a Coworker: Why Local AI Coding Agents Are Eating the Cloud
I uninstalled my cloud AI plugin yesterday. Nothing broke. antirez just shipped something called ds4 that runs DeepSeek V4 Flash at 40+ tok/s on my 128GB MacBook. Same model. No network. No API key. No wondering which server my company's code just got shipped off to.
This isn't a recommendation. It's a field report.
Three projects hit GitHub Trending the same weekend: DeepSeek-Reasonix (a long-running terminal coding agent), ds4 (antirez's local inference engine), and Wafer's benchmark of Kimi K3 on AMD MI355X. Different people, different repos, no coordination. Read them back to back and they tell one story: AI coding tools are moving from "renting someone else's brain" to "a coworker that lives on your hard drive."
Three Pieces, One Thread
Piece One: ds4 — Getting the Model to Run on Your Machine
antirez built Redis. His software has a signature: do one thing, do it ruthlessly well. ds4 (DwarfStar) follows the pattern. It's a DeepSeek V4 inference engine. Not a general GGUF loader, not a platform, just an engine.
What it actually does:
-
DeepSeek V4 and GLM 5.2 only. Model loading, prompt rendering, tool calls, KV cache, HTTP server: all of it vertically optimized for exactly two model families. No abstraction tax.
-
Metal, CUDA, ROCm. Pick your hardware and it runs. MacBook, DGX Spark, Strix Halo, whatever's on the desk.
-
Aggressive quantization that actually works. MoE expert layers get crushed to 2-bit (IQ2_XXS). Shared experts, projections, and routing stay at full precision. Model size drops dramatically, coding quality doesn't. antirez tested this himself: the 2-bit quant of DeepSeek Flash hits the same tool-calling accuracy as full precision in agent workloads.
-
Glue machines together. Two M5 Maxes over RDMA run 4-bit Flash with tensor parallelism. Need more RAM than any one box has? Pipeline parallelism lets you stack them.
One line in the README sold me:
"Using the CUDA multi-GPU support, you can turn a server with old-ish CUDA cards, no longer supported for new models by vLLM, into a multi-user LLM server for your company. We tested with 8x L40S and got 120 tok/s aggregate generation, 2000 tok/s prefill."
Those dusty L40S cards vLLM abandoned? ds4 brings them back from the dead. 120 tok/s concurrent is plenty for a small team.
Piece Two: Reasonix — Keeping the Agent Alive in Your Terminal
You've got a model running. Now you need something to talk to it.
DeepSeek-Reasonix ships as a single Go binary. Run npm install -g reasonix and you're done: no Python venv, no Docker, no CUDA toolkit. Its entire design philosophy orbits one idea, prefix-cache stability.
DeepSeek's API charges only for incremental tokens when your prompt prefix stays the same. Reasonix is engineered around that fact. It injects a compact environment summary at startup, prunes stale tool output before compaction, and holds prefix-cache hit rates high across hours-long sessions. Leave it running all day and your token costs stay flat instead of climbing with the clock.
Plugins ride on top. External tools connect over stdio JSON-RPC (MCP-compatible), and the VS Code extension, desktop app, and CLI all share the same engine underneath.
Which reminds me of tmux. Not because it looks like tmux, but because both solve the same class of problem: keeping something alive in a terminal beats spawning it fresh every time.
Piece Three: Hardware Economics Are Redrawing the Battle Lines
Wafer's benchmark brings the hard numbers. They ran Kimi K3, a 2.8 trillion parameter model, on AMD MI355X. Kimi K3 needs over 1.5TB of VRAM just for weights plus a 1M-token KV cache.
So why AMD? The MI355X packs 288GB HBM, same as a B300, at 2.4× less per GPU. And here's the kicker: a single B200 node (8×192GB) can't even fit Kimi K3. You need two nodes in TP16, which means every decode step pays a cross-node RoCE v2 round trip. The MI355X fits the whole thing in one node and skips that tax entirely.
The numbers:
| Metric | MI355X (TP8) | B200 (TP16, 2-node) | B300 (TP8) |
|---|---|---|---|
| Single-stream decode | 118 tok/s | 90 tok/s | 172 tok/s |
| Peak aggregate | 952 tok/s | 498 tok/s | 1,568 tok/s |
| Perf per dollar | 48 tok/s/$ | 7 tok/s/$ | 33 tok/s/$ |
48 versus 7. That's not a rounding error, that's a 7× gap in throughput per dollar.
Wafer's team also fixed two ROCm ecosystem bugs on the way. First, the speculative decode verifier crashed because ROCm's build was missing a top_k_renorm_prob kernel, so they rewrote it in three lines of PyTorch. Second, the AITER MLA prefill kernel rejected Kimi K3's head count (12 heads per rank, where the kernel expected 4, 8, or multiples of 16), so they zero-padded to 16 and sliced the result back down. Prefill got 2–3× faster.
Both bugs point at the same reality. AMD's software story still has potholes, but somebody's filling them. Wafer's own take: "this time it certainly did not require custom kernels." Progress.
Why These Three Belong in the Same Conversation
Read separately, ds4 is an inference engine, Reasonix is an agent framework, and Wafer's post is a hardware blog. Nothing connects them. Read together, they answer one question: what's still missing from a fully local AI development workflow?
Answer: nothing.
- Hardware: One MI355X, or a 128GB MacBook, or a handful of retired L40S cards. All viable.
- Inference: ds4 squeezes DeepSeek Flash to 2-bit without losing tool-calling reliability.
- Agent: Reasonix keeps the model resident in your terminal, and prefix-caching keeps costs flat.
- Economics: AMD is using HBM capacity to undercut NVIDIA's pricing, and a 7× gap per dollar is hard for whoever signs the PO to ignore.
None of this is a five-year forecast. You could have it running Saturday afternoon.
The Fine Print
Wafer sells AMD inference. Their benchmark runs on their own hardware, and "performance per dollar" happens to be the conclusion their business needs. I'm not calling the data fake. I'm saying your workload will produce different numbers. Run your own benchmark.
2-bit quantization isn't free. antirez's strategy is genuinely clever, quantizing only MoE experts and leaving shared components untouched. But what he tested was agent tool-calling, not novel writing. If your work is long-form or creative generation, nobody has checked whether those quantization artifacts compound where coding benchmarks never look.
Local doesn't mean maintenance-free. ds4's README says "beta quality" right there in the text, and Reasonix is iterating fast. Move your workflow local and you now own model updates, quantization versioning, and hardware compatibility. Your cloud API was quietly handling every bit of that.
What You Can Do Today
-
Got a Mac (M-series, 64GB+)? Spend 20 minutes with ds4:
git clone https://github.com/antirez/ds4 cd ds4 && ./download_model.sh q2-imatrix make && ./ds4-serverFind out what local inference latency actually feels like.
-
Code in a terminal? Install Reasonix:
npm i -g reasonix reasonix setup reasonixRun one full coding session and compare it to whatever cloud plugin you're paying for now.
-
Manage GPU budgets? Send Wafer's table to procurement. B200 at 7 tok/s/$ against MI355X at 48 tok/s/$ belongs in your next purchase-order conversation.
Pick one. It'll cost you an afternoon. Then decide whether that cloud plugin is still earning its seat.
✨ Written by DeepSeek, reviewed and polished by Claude.
Sources: