
A 35B language model running on an iPhone using only 1–2.5 GB of peak memory. No cloud. No remote server. No desktop GPU. Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device.

Founder, edge0.ai | On-device foundation models. Powerful intelligence, finally on your device. | MIT TR35 linktr.ee/samuelzeng

A 35B language model running on an iPhone using only 1–2.5 GB of peak memory. No cloud. No remote server. No desktop GPU. Today, we’re open-sourcing Edge0 — a framework for running large AI models fully on-device.

Great!
Edge0-145B driving 6 other models at 2-3 streams each, thinking on, 64~128k context per, writing code from scratch. Was trying to do 40 streams but Edge0 will not delegate more

Anyone want an on-device video generation model? Our team is working on one.

Update: Edge0 now runs on Mac 🍎 Many of you requested macOS support. Edge0 Chat for Apple Silicon is now live, with Edge0-145B and Edge0-120B running locally via Metal. How to get started: - Download edge0-chat-0.2.0-macos-arm64… - Get the model weights from Hugging Face - Load the model and start chatting, or plug into your favorite coding agent—fully offline Requires macOS 11+ on Apple Silicon. 👇 Download github.com/Edge0-AI/Edge0/re… 👇 GitHub Homepage github.com/Edge0-AI/Edge0

How to run Edge0-145B on your own laptop in 6 steps. Our chief architect Bin walks you through it 👇 1️⃣ Download Edge0 Chat from our GitHub releases 2️⃣ Download the Edge0-145B weights from Hugging Face 3️⃣ Load the model in the app 4️⃣ Start chatting fully local (toggle thinking on/off) 5️⃣ Check live stats: ~1,050 tok/s prefill, ~77 tok/s decode 6️⃣ Turn on the local service, then paste the base URL into your favorite vibe coding tool

Both models are open today: the weights and the engine. ⛰️ Edge0-145B: flagship — 86.5 on Terminal-Bench 2.1 🪶 Edge0-120B: lighter version Runs on your laptop or desktop. Serves an OpenAI- and Anthropic-compatible API, so you can point your coding agent at localhost and keep working offline. 🤗 145B: huggingface.co/Edge0/Edge0-1… 🤗 120B: huggingface.co/Edge0/Edge0-1… 💻 Engine: github.com/Edge0-AI/Edge0 💻 App: github.com/Edge0-AI/Edge0/re… Build something with it? Reply with what you made—we’ll share the best ones.

How do we run a 145B model on a laptop or consumer GPU? The short answer: we don't keep all 145B parameters on the GPU at once. Edge0-145B is a sparse MoE model with 512 experts per layer. Only 10 are activated for each token, which works out to roughly 6.6B active parameters. The key is our latent MoE architecture. Attention operates at the full width of 2560. Before a token reaches the experts, a shared projection compresses it to 1792 dimensions. Another projection expands it back afterward. This means the experts themselves can stay small, about 1.6 MB each. We keep a few thousand frequently used experts on the GPU and stream the rest from system RAM as needed. So GPU memory usage depends largely on how many experts we keep loaded, rather than the model's full parameter count. The model is also trained natively in 4-bit, with a built-in draft head to speed up decoding.

Local vibe coding #4: PCB design This time we gave Edge0-145B a different task: design a circuit board, then animate how it gets made. It placed the components and routed both layers in KiCad, brought the board into Unreal Engine 5, set up the cameras and lighting, and rendered a fabrication and assembly animation. PCB design and 3D animation are usually two separate workflows, often handled by different people. Here, one model handled both on a single machine. And none of the schematics or layouts had to leave the device. For hardware teams working with sensitive designs, that last part matters a lot. Cloud tokens: 0. API cost: $0.

Local vibe coding #3: video production Can a flying Roadster actually work? We gave Edge0-145B one prompt to find out. It worked through the physics, wrote a script, recreated the car in 3D, and put together a 94-second explainer. All on a laptop, without calling a cloud model. Cloud tokens: 0. API cost: $0.

Local vibe coding #2: game worlds One prompt, five reference images, and no internet. We asked Edge0-145B to build a medieval city for a fantasy RPG. It built the whole thing from scratch: red-tiled roofs, a market square, a Gothic cathedral with twin towers, and even a vaulted interior with stained-glass windows. The tricky part was the camera path. We fixed it in advance, so the geometry had to actually work. One misplaced wall or column, and the camera would clip right through it. Cloud tokens: 0. API cost: $0.

Local vibe coding #1: 3D modeling One prompt. Wi-Fi off. We asked Edge0-145B to build Manhattan in Blender. It wrote the Python code, generated thousands of buildings on a street grid, added the skyline and rivers on both sides, and rendered a 20-second flythrough. It even checked its own renders, caught mistakes, and went back to fix them. Cloud tokens: 0. API cost: $0.

Light and fast: 100B-class models fit on a laptop. - Edge0-145B: 29GB peak memory; 72.8 avg across 6 benchmarks - Qwen3.8-Flash-Next: ~70GB; 71.7 - DeepSeek-V4-Flash: ~87GB; 70.1 Same intelligence as models that need 3× more memory.

Frontier-level agentic coding, fully on-device. Across six major benchmarks, Edge0-145B performs on par with leading open cloud models such as DeepSeek-V4-Flash. Its edge is agentic coding: - 64.6 on DeepSWE 1.1 - 86.5 on Terminal-Bench 2.1 It’s ahead of comparable open models on both. On Terminal-Bench, IFBench, and AA-LCR, it also outscores the closed cloud model Claude Opus 4.6 (Max).

For 18 months, every lab’s on-device models stayed under 40B. Today, we pushed on-device AI to 145B parameters.

Introducing Edge0-145B and Edge0-120B, @Edge0_AI’s first device-native 100B-class foundation models with open weights. Cloud-level intelligence on your laptop. Free, offline, and fast enough for real vibe coding. - Smart: beats Claude Opus 4.6 on Terminal-Bench 2.1 (86.5% vs. 78.2%) - Light: needs just 29G memory, 1/3 of a standard setup (100G+) - Fast: decodes 75+ toks/s, prefills 1050+ toks/s on RTX Spark and Mac. - Steady: handles 128K–256K tokens of context without slowing down, even on long tasks Frontier-class agentic coding. No cloud API. No token bill. Your code and data never leave your machine. Open weights are live on @huggingface. Run it locally or plug it into your favorite coding agent.

The SFT training code is also included in this version. Enjoy! github.com/Edge0-AI/Audio8_T…