Edge0-145B: a 145B coding agent that runs on one consumer GPU
Edge0 released Edge0-145B, an open-weights 145-billion-parameter latent MoE model for local coding and agent work. It activates about 10B parameters per token, holds a 256K context, and the vendor reports 121 tok/s agent decode on a single RTX 5090 D. Weights, a C++ inference engine, and a desktop app are all out, under a community license that restricts commercial assistant products.
Edge0 released Edge0-145B this week, and the claim is worth your attention: a 145-billion-parameter model, built for coding and agent work, running on a single consumer GPU or an Apple Silicon Mac, with weights you can download today. The vendor reports 121.2 tokens per second of agent decode on one RTX 5090 D 24GB and a measured 29GB peak memory on a laptop-class setup, with benchmark scores in the same table as frontier cloud models. If those numbers hold up under independent testing, the economics of running a serious coding agent change: no token bills, no data leaving the machine, no network dependency. All performance and benchmark figures below are vendor-reported; I have not run this model myself.
What Edge0-145B actually is
Edge0-145B is a latent mixture-of-experts model derived from Qwen3.8-Flash-Next, but rebuilt rather than re-quantized. Edge0 replaced the routed expert layer in all 48 layers: each token is projected down to a narrow latent width (1,792 on the 145B), the experts compute there, and the result is projected back to the full 2,560 width. The model was then post-trained on Edge0's own data and trained natively at 4 bits, so this is not an after-the-fact quantization.
The headline numbers from the model card:
- 145B total parameters, 10.12B active per token
- 48 layers, 512 experts per layer, 10 routed plus 1 shared active per token
- 256K context, with the KV cache in int8 and 32K tokens resident
- A 1.26B-parameter MTP draft head for speculative decoding
- Two builds: Q3_K at 40.81 GiB of weights, NVFP4 at 47.79 GiB
There is also a 120B sibling (latent width 1,280, 60.4B routed expert parameters) that trades some quality for speed: the vendor reports 144.6 tok/s agent decode on the same RTX 5090 D.
The rebuild cut routed expert parameters by 30% against the Qwen base (84.6B versus 120.8B) and shrank each expert to a fixed 1.63MB record. On a constrained budget of 16GB VRAM plus 20GB system RAM, Edge0 reports 36.8 tok/s chat decode for the 145B where the Qwen base manages 18.5, roughly a 2x gain from the architecture change alone.
The speed and memory numbers, all vendor-reported
The GitHub README publishes a full throughput table. The rows that matter for most builders:
| Machine | Build | 16K prefill | Chat decode | Agent decode |
|---|---|---|---|---|
| RTX 5090 D 24GB | Q3_K | 3,554.9 | 99.7 | 121.2 |
| RTX 4090 24GB | Q3_K | 3,179.4 | 69.9 | 91.7 |
| RTX Spark N1X 60GiB | NVFP4 | 922.4 | 40.1 | 46.0 |
| Apple M2 Ultra (Metal) | Q3_K | 800.3 | 60.9 | 57.0 |
| 16GB VRAM + 20GB RAM (constrained) | Q3_K | 335 | 36.8 | 44.9 |
Two honest caveats on that table. First, the constrained row is the same RTX 5090 D artificially capped, not a cheap card; it shows the engine degrades gracefully, not that a mid-range setup is fast. Second, the launch blog's laptop claims (80+ tok/s decode, 29GB peak, about 120W) describe a high-end configuration. The measured Apple Silicon row in the README is an M2 Ultra at roughly 57 to 61 tok/s, so treat the laptop numbers as best-case until someone reproduces them.
Benchmarks: competitive on agentic coding, with the usual asterisks
Edge0's own evaluation table puts the 145B ahead of its Qwen base and DeepSeek-V4-Flash-0731 on the agentic coding benchmarks:
| Model | DeepSWE 1.1 | Terminal-Bench 2.1 | GPQA-Diamond | IFBench |
|---|---|---|---|---|
| Edge0-145B | 64.6 | 86.52 | 90.4 | 80.6 |
| Qwen3.8-Flash-Next | 58.7 | 83.3 | 91.7 | 81.3 |
| DeepSeek-V4-Flash-0731 | 54.4 | 82.7 | 90.8 | 79.2 |
| Claude-Opus-4.6 (Max) | no public score | 78.2 | 91.3 | 62.5 |
The mean across six benchmarks is 72.8 for Edge0-145B against 71.7 for Qwen3.8-Flash-Next, at what Edge0 says is about 40% of the memory footprint (29GB measured versus roughly 70GB estimated for a standard 4-bit deployment of the Qwen model).
Everything in that table comes from Edge0's internal evaluations, and some competitor scores are cited from third-party evaluators. The pattern to note is not the exact scores but the shape: the model's advantage concentrates in agentic coding and terminal work, which matches what the architecture was rebuilt for. Independent reproduction is what to watch for next.
How a 145B model fits on one card
The engineering story is the interesting part, and it comes in four layers.
#### Latent MoE keeps expert records tiny
Because experts compute at the 1,792 latent width instead of full width, each expert is a fixed 1.63MB record. The whole expert pool is 37.32 GiB on disk in Q3_K. The engine can therefore hold a few thousand experts on the card and stream the rest, at a cost that tracks how many experts are resident rather than how big the model is.
#### Three-tier loading overlaps storage, RAM and VRAM
Hot experts stay in VRAM, an expert library of roughly 40GB sits in system memory, and vocabulary data lives on the SSD read on demand. Reads and computation overlap across the three tiers. The 145B model on a 24GB card works because the engine keeps disjoint expert sets in VRAM and RAM (23.14 GiB plus 31.28 GiB for Q3_K, nothing counted twice) and can fall further back to SSD with the mmap mode when RAM is tight.
#### A router that predicts ahead
The 35B technical report (arXiv 2609.18063) describes the trick underneath: routing is predicted a step ahead, so the engine prefetches the experts it is about to need instead of stalling on loads. The roadmap pushes this further, moving routing prediction from per-position to once per block.
#### Speculative decoding spreads the weight reads
A 1.26B draft model proposes several tokens per verify step, so each expensive pass over the weights produces more output. That is part of how agent decode (121.2 tok/s) beats chat decode (99.7) on the same hardware.
Running it yourself
The tooling is unusually complete for a fresh release. edge0-infer is a single C++ engine (GGML-based, Apache-2.0) with one API across platforms. Two commands cover most use:
edge0-infer serve --model /path/to/Edge0-145B-q3k
edge0-infer chat --model /path/to/Edge0-145B-q3k
The serve command exposes OpenAI- and Anthropic-compatible endpoints on localhost:8000, so existing agent harnesses can point at it with a base-URL change. There is also an Edge0 Chat desktop app for Windows and macOS that downloads models from Hugging Face or ModelScope in-app, and an optional vision encoder (mmproj-BF16.gguf) that adds image input.
Platform support for the 145B and 120B: Linux or Windows with an NVIDIA GPU on CUDA 13, or macOS on Apple Silicon via Metal (Q3_K only). No iOS or Android for this tier. A second GPU can hold the overflow experts without NVLink or P2P, which is a practical upgrade path if one 24GB card feels tight.
The license is the catch
The framework is Apache-2.0 and the engine fork keeps its MIT license, but the model weights sit under the Edge0 Community Model License 1.0, and it is not a permissive free-for-all. Personal use, research, education, modification, noncommercial sharing, internal use, and commercial use of the model's outputs are all allowed. Three things require separate written permission from Edge0:
- Selling or paid distribution of the model itself, including commercial bundles whose main value is model access
- Commercial model services offering inference or fine-tuning for fees, subscriptions, or advertising
- Independent commercial coding or office assistants, whether inference runs on your server or on the user's device
That third clause matters for exactly the audience reading this. Building a commercial coding assistant on these weights is not covered by the default license; you need to talk to Edge0 first. Internal tooling at your company is covered. The Qwen upstream license also still applies on top.
What I could not verify
Every performance figure, benchmark score, and demo in this article comes from Edge0's blog, README, and model cards. Nobody outside the company has published independent benchmark runs yet, the demo videos are vendor-produced, and the portfolio-scale claims (15 million downloads, 5 million production devices, 110,000 customer organizations) are company figures I cannot check. The laptop-class power and speed claims in particular deserve skepticism until reproduced. What is verifiable today: the weights are actually downloadable, the engine code is public with an active repository (about 12,000 stars), the license text says what I described, and the architecture details are documented in unusual depth, including exact memory-residency breakdowns that invite reproduction.
Frequently asked questions
Can Edge0-145B really run on one consumer GPU?
Yes, with precision about what "run" means. The vendor's own table shows 91.7 tok/s agent decode on an RTX 4090 24GB and 121.2 on an RTX 5090 D, by splitting the expert pool across VRAM and system RAM and streaming the rest. You need the RAM to match: the Q3_K build holds about 31GB of experts in host memory on a 24GB card.
What hardware do I need at minimum?
Edge0 tested down to a 16GB VRAM plus 20GB system RAM budget and still got 36.8 tok/s chat decode, but that was a capped RTX 5090 D, not a budget card. Realistically: an NVIDIA GPU with 24GB, 64GB or more of system RAM, and a fast SSD. Apple Silicon Macs run the Q3_K build over Metal, with an M2 Ultra measured at roughly 57 to 61 tok/s.
Is Edge0-145B good enough to replace a cloud coding model?
On the vendor's own agentic coding benchmarks it beats its Qwen base and DeepSeek-V4-Flash and sits near Claude Opus 4.6 on some rows, but those are internal numbers and cloud models have their own advantages on long-horizon reliability. The honest answer: promising enough to evaluate on your own codebase, not proven enough to cancel your API keys.
How is Edge0-145B different from just quantizing Qwen3.8-Flash-Next?
The architecture changed, not just the precision. Edge0 rebuilt the routed expert layer in all 48 layers into a latent MoE, cutting routed expert parameters by 30% and each expert to a 1.63MB record, then post-trained the result natively at 4 bits. On the same constrained memory budget the vendor reports double the decode speed of the Qwen base.
Is Edge0-145B free for commercial use?
Internal use and commercial use of the model's outputs are allowed. Selling the model, offering it as a paid inference or fine-tuning service, or building an independent commercial coding or office assistant on it all require written permission from Edge0. The Qwen upstream license applies as well.
Does Edge0-145B work offline?
Yes for the core model: weights, engine, and the desktop app's inference all run locally with no account or quota. Anything that needs current information or external services obviously still needs a connection.
What is the difference between the 145B and the 120B?
Same architecture family, different latent width (1,792 versus 1,280) and a smaller expert pool on the 120B (60.4B versus 84.6B routed parameters). The 120B is meaningfully faster (144.6 tok/s agent decode on an RTX 5090 D) and scores lower on coding benchmarks (51.3 versus 64.6 on DeepSWE 1.1). If latency matters more than peak quality, the 120B is the practical pick.
Sources
Related reading
- LongCat-2.5: Meituan open-sources a 1.6T agent model under MIT, trained and served without Nvidia
Meituan's LongCat team released the weights for LongCat-2.5, a 1.6-trillion-parameter mixture-of-experts model with about 48B active parameters per token, native image understanding, a 512K-token context, and an MIT license. Training and the reference deployment run entirely on domestic Chinese accelerators, with W4A4 INT4 quantization that the team says roughly matches full-precision accuracy. Vendor benchmarks put it near Claude Opus 5 on agentic coding, below it on terminal tasks. Self-hosting needs a serious cluster.
- SenseNova Loop-9B: SenseTime's 9B model that runs its layers twice
SenseTime's SenseNova Loop-9B reuses 24 shared layers to get 56 layer computations out of roughly 9B parameters, claims parity with Qwen3.8-27B on vendor benchmarks, and runs 256K context in about 15 GB of GPU memory. What builders should know before downloading.
- GPT-6.2 Astra and Sol: OpenAI's answer to the price-performance war
OpenAI released GPT-6.2 Astra and Sol on November 17, 2026: a frontier model for the hardest professional work at $8 per million input tokens, and an everyday model holding GPT-6.1 Sol's prices. The launch leans on cost-per-task claims across legal, science, coding and 3D benchmarks, ships both models in ChatGPT Work, Codex and the API, and includes a rare disclosure: Astra is OpenAI's first external model to hit the High threshold for AI self-improvement.
The build log
Follow the work.
New essays and prototypes, in your feed reader. Email subscriptions are paused while I connect a working delivery service. No email is collected here.
Follow via RSS