# Edge0-145B: a 145B coding agent that runs on one consumer GPU


Edge0 released Edge0-145B this week, and the claim is worth your attention: a 145-billion-parameter model, built for coding and agent work, running on a single consumer GPU or an Apple Silicon Mac, with weights you can download today. The vendor reports 121.2 tokens per second of agent decode on one RTX 5090 D 24GB and a measured 29GB peak memory on a laptop-class setup, with benchmark scores in the same table as frontier cloud models. If those numbers hold up under independent testing, the economics of running a serious coding agent change: no token bills, no data leaving the machine, no network dependency. All performance and benchmark figures below are vendor-reported; I have not run this model myself.

## What Edge0-145B actually is

Edge0-145B is a latent mixture-of-experts model derived from Qwen3.8-Flash-Next, but rebuilt rather than re-quantized. Edge0 replaced the routed expert layer in all 48 layers: each token is projected down to a narrow latent width (1,792 on the 145B), the experts compute there, and the result is projected back to the full 2,560 width. The model was then post-trained on Edge0's own data and trained natively at 4 bits, so this is not an after-the-fact quantization.

The headline numbers from the model card:

- 145B total parameters, 10.12B active per token
- 48 layers, 512 experts per layer, 10 routed plus 1 shared active per token
- 256K context, with the KV cache in int8 and 32K tokens resident
- A 1.26B-parameter MTP draft head for speculative decoding
- Two builds: Q3_K at 40.81 GiB of weights, NVFP4 at 47.79 GiB

There is also a 120B sibling (latent width 1,280, 60.4B routed expert parameters) that trades some quality for speed: the vendor reports 144.6 tok/s agent decode on the same RTX 5090 D.

The rebuild cut routed expert parameters by 30% against the Qwen base (84.6B versus 120.8B) and shrank each expert to a fixed 1.63MB record. On a constrained budget of 16GB VRAM plus 20GB system RAM, Edge0 reports 36.8 tok/s chat decode for the 145B where the Qwen base manages 18.5, roughly a 2x gain from the architecture change alone.

## The speed and memory numbers, all vendor-reported

The GitHub README publishes a full throughput table. The rows that matter for most builders:

| Machine | Build | 16K prefill | Chat decode | Agent decode |
| --- | --- | --- | --- | --- |
| RTX 5090 D 24GB | Q3_K | 3,554.9 | 99.7 | 121.2 |
| RTX 4090 24GB | Q3_K | 3,179.4 | 69.9 | 91.7 |
| RTX Spark N1X 60GiB | NVFP4 | 922.4 | 40.1 | 46.0 |
| Apple M2 Ultra (Metal) | Q3_K | 800.3 | 60.9 | 57.0 |
| 16GB VRAM + 20GB RAM (constrained) | Q3_K | 335 | 36.8 | 44.9 |

Two honest caveats on that table. First, the constrained row is the same RTX 5090 D artificially capped, not a cheap card; it shows the engine degrades gracefully, not that a mid-range setup is fast. Second, the launch blog's laptop claims (80+ tok/s decode, 29GB peak, about 120W) describe a high-end configuration. The measured Apple Silicon row in the README is an M2 Ultra at roughly 57 to 61 tok/s, so treat the laptop numbers as best-case until someone reproduces them.

## Benchmarks: competitive on agentic coding, with the usual asterisks

Edge0's own evaluation table puts the 145B ahead of its Qwen base and DeepSeek-V4-Flash-0731 on the agentic coding benchmarks:

| Model | DeepSWE 1.1 | Terminal-Bench 2.1 | GPQA-Diamond | IFBench |
| --- | --- | --- | --- | --- |
| Edge0-145B | 64.6 | 86.52 | 90.4 | 80.6 |
| Qwen3.8-Flash-Next | 58.7 | 83.3 | 91.7 | 81.3 |
| DeepSeek-V4-Flash-0731 | 54.4 | 82.7 | 90.8 | 79.2 |
| Claude-Opus-4.6 (Max) | no public score | 78.2 | 91.3 | 62.5 |

The mean across six benchmarks is 72.8 for Edge0-145B against 71.7 for Qwen3.8-Flash-Next, at what Edge0 says is about 40% of the memory footprint (29GB measured versus roughly 70GB estimated for a standard 4-bit deployment of the Qwen model).

Everything in that table comes from Edge0's internal evaluations, and some competitor scores are cited from third-party evaluators. The pattern to note is not the exact scores but the shape: the model's advantage concentrates in agentic coding and terminal work, which matches what the architecture was rebuilt for. Independent reproduction is what to watch for next.

## How a 145B model fits on one card

The engineering story is the interesting part, and it comes in four layers.

#### Latent MoE keeps expert records tiny

Because experts compute at the 1,792 latent width instead of full width, each expert is a fixed 1.63MB record. The whole expert pool is 37.32 GiB on disk in Q3_K. The engine can therefore hold a few thousand experts on the card and stream the rest, at a cost that tracks how many experts are resident rather than how big the model is.

#### Three-tier loading overlaps storage, RAM and VRAM

Hot experts stay in VRAM, an expert library of roughly 40GB sits in system memory, and vocabulary data lives on the SSD read on demand. Reads and computation overlap across the three tiers. The 145B model on a 24GB card works because the engine keeps disjoint expert sets in VRAM and RAM (23.14 GiB plus 31.28 GiB for Q3_K, nothing counted twice) and can fall further back to SSD with the mmap mode when RAM is tight.

#### A router that predicts ahead

The 35B technical report (arXiv 2609.18063) describes the trick underneath: routing is predicted a step ahead, so the engine prefetches the experts it is about to need instead of stalling on loads. The roadmap pushes this further, moving routing prediction from per-position to once per block.

#### Speculative decoding spreads the weight reads

A 1.26B draft model proposes several tokens per verify step, so each expensive pass over the weights produces more output. That is part of how agent decode (121.2 tok/s) beats chat decode (99.7) on the same hardware.

## Running it yourself

The tooling is unusually complete for a fresh release. `edge0-infer` is a single C++ engine (GGML-based, Apache-2.0) with one API across platforms. Two commands cover most use:

```bash
edge0-infer serve --model /path/to/Edge0-145B-q3k
edge0-infer chat --model /path/to/Edge0-145B-q3k
```

The serve command exposes OpenAI- and Anthropic-compatible endpoints on localhost:8000, so existing agent harnesses can point at it with a base-URL change. There is also an Edge0 Chat desktop app for Windows and macOS that downloads models from Hugging Face or ModelScope in-app, and an optional vision encoder (mmproj-BF16.gguf) that adds image input.

Platform support for the 145B and 120B: Linux or Windows with an NVIDIA GPU on CUDA 13, or macOS on Apple Silicon via Metal (Q3_K only). No iOS or Android for this tier. A second GPU can hold the overflow experts without NVLink or P2P, which is a practical upgrade path if one 24GB card feels tight.

## The license is the catch

The framework is Apache-2.0 and the engine fork keeps its MIT license, but the model weights sit under the Edge0 Community Model License 1.0, and it is not a permissive free-for-all. Personal use, research, education, modification, noncommercial sharing, internal use, and commercial use of the model's outputs are all allowed. Three things require separate written permission from Edge0:

- Selling or paid distribution of the model itself, including commercial bundles whose main value is model access
- Commercial model services offering inference or fine-tuning for fees, subscriptions, or advertising
- Independent commercial coding or office assistants, whether inference runs on your server or on the user's device

That third clause matters for exactly the audience reading this. Building a commercial coding assistant on these weights is not covered by the default license; you need to talk to Edge0 first. Internal tooling at your company is covered. The Qwen upstream license also still applies on top.

## What I could not verify

Every performance figure, benchmark score, and demo in this article comes from Edge0's blog, README, and model cards. Nobody outside the company has published independent benchmark runs yet, the demo videos are vendor-produced, and the portfolio-scale claims (15 million downloads, 5 million production devices, 110,000 customer organizations) are company figures I cannot check. The laptop-class power and speed claims in particular deserve skepticism until reproduced. What is verifiable today: the weights are actually downloadable, the engine code is public with an active repository (about 12,000 stars), the license text says what I described, and the architecture details are documented in unusual depth, including exact memory-residency breakdowns that invite reproduction.

## Frequently asked questions

### Can Edge0-145B really run on one consumer GPU?

Yes, with precision about what "run" means. The vendor's own table shows 91.7 tok/s agent decode on an RTX 4090 24GB and 121.2 on an RTX 5090 D, by splitting the expert pool across VRAM and system RAM and streaming the rest. You need the RAM to match: the Q3_K build holds about 31GB of experts in host memory on a 24GB card.

### What hardware do I need at minimum?

Edge0 tested down to a 16GB VRAM plus 20GB system RAM budget and still got 36.8 tok/s chat decode, but that was a capped RTX 5090 D, not a budget card. Realistically: an NVIDIA GPU with 24GB, 64GB or more of system RAM, and a fast SSD. Apple Silicon Macs run the Q3_K build over Metal, with an M2 Ultra measured at roughly 57 to 61 tok/s.

### Is Edge0-145B good enough to replace a cloud coding model?

On the vendor's own agentic coding benchmarks it beats its Qwen base and DeepSeek-V4-Flash and sits near Claude Opus 4.6 on some rows, but those are internal numbers and cloud models have their own advantages on long-horizon reliability. The honest answer: promising enough to evaluate on your own codebase, not proven enough to cancel your API keys.

### How is Edge0-145B different from just quantizing Qwen3.8-Flash-Next?

The architecture changed, not just the precision. Edge0 rebuilt the routed expert layer in all 48 layers into a latent MoE, cutting routed expert parameters by 30% and each expert to a 1.63MB record, then post-trained the result natively at 4 bits. On the same constrained memory budget the vendor reports double the decode speed of the Qwen base.

### Is Edge0-145B free for commercial use?

Internal use and commercial use of the model's outputs are allowed. Selling the model, offering it as a paid inference or fine-tuning service, or building an independent commercial coding or office assistant on it all require written permission from Edge0. The Qwen upstream license applies as well.

### Does Edge0-145B work offline?

Yes for the core model: weights, engine, and the desktop app's inference all run locally with no account or quota. Anything that needs current information or external services obviously still needs a connection.

### What is the difference between the 145B and the 120B?

Same architecture family, different latent width (1,792 versus 1,280) and a smaller expert pool on the 120B (60.4B versus 84.6B routed parameters). The 120B is meaningfully faster (144.6 tok/s agent decode on an RTX 5090 D) and scores lower on coding benchmarks (51.3 versus 64.6 on DeepSWE 1.1). If latency matters more than peak quality, the 120B is the practical pick.

## Sources

- [Edge0-145B launch blog post](https://edge0.ai/blog/introducing-edge0-145b-frontier-ai-capabilities-on-your-own-computer)
- [Edge0 GitHub repository (engine, benchmarks, license)](https://github.com/Edge0-AI/Edge0)
- [Edge0-145B Q3_K model card on Hugging Face](https://huggingface.co/Edge0/Edge0-145B-q3k)
- [Edge0-145B NVFP4 weights on Hugging Face](https://huggingface.co/Edge0/Edge0-145B-NVFP4)
- [arXiv 2609.18063: The Other Half of the Memory Wall (35B technical report)](https://arxiv.org/abs/2609.18063)
