←
Mesh LLM14 posts
Mesh LLM avatar

Mesh LLM

@Mesh_LLM

Distributed inference for the people. Private and public mesh, sharing hardware for inference. github.com/mesh-llm/mesh-llm

◷ Joined October 2026▧ 2 Photos and videos
Mesh LLM avatar
Mesh LLM@Mesh_LLM·Nov 14

Mesh aims to support as many hardware platforms and OSes as possible in a uniform way. some is easier than others. Here is Nick casing the GPU isles to get some amd to make sure ROCm is well supported.

Post attachmentPost attachment
Mesh LLM avatar
Mesh LLM@Mesh_LLM·Nov 13

coming up real soon now: document extractor plugin (read any kind of docs even if underlying model can't), faster routing, proof of inference (trust the model is really being served) and much more...

Mesh LLM avatar
Mesh LLM@Mesh_LLM·Nov 13

New release is out, boringly named 0.79. Some goodies: speech models DFlash speculative decoding /v1/systemone decision models MoE experts in system RAM; on-demand model loading Windows background-service installation. plugins: power measurement, telemetry and payments support

ThomPete avatar
⟳ Mesh LLM retweeted
ThomPete@Hello_World·Nov 2

Full writeup on shared compute coming soon. Until then, here’s another look at what plugins can do. Left (Machine 1): Agent Palmer has no LLM attached. It runs entirely on shared compute from the relay. Right (Machine 2): A regular MacBook running a local @QwenDevs model, sharing its compute with anyone on the network who needs it. The little black square: Machine 2’s floating window. It shows live activity even when you’re outside the shared compute view. Bonus: I added USB support to an ESP32 unit that stays synced with Machine 2. @Mesh_LLM handles all of it behind the scenes. IMO shared compute is still extremely underutilized. Think of it as blue-collar AI: it runs quietly on your relay and handles background tasks and batch jobs, powered entirely by your community’s machines. And since @buzzdotxyz is open source, you can run all of this on your own servers. That’s one small step for buzz, but one giant step toward sovereignty for all mankind.

1:27
Mesh LLM avatar
Mesh LLM@Mesh_LLM·Oct 29

New Mesh plugins for Apple Silicon: ⚡ Uzu: high-throughput Metal inference for Qwen 3.8 & more 🧠 DwarfStar: run huge DeepSeek V4 models locally 💾 Slotstream: SSD-backed MoE for Macs with less RAM github.com/Mesh-LLM/uzu-runt… github.com/Mesh-LLM/ds4-plug… github.com/Mesh-LLM/slotstre…

Mat Balez avatar
⟳ Mesh LLM retweeted
Mat Balez@matbalez·Oct 27

this was fun to build - fork of signal - agent built with GDK - using open weights model - served via e2ee mesh-llm - via private GPU TEE - making private search tool calls - paid for with ₿ over LN a cool stack of permissionless & open source tech all working together!

An agent inside Signal using decentralized compute paid for with bitcoin @moneyball demos a fork of Signal built by @matbalez using @Mesh_LLM and @lexeapp ↓ Timestamps: 0:00 - Demo: agent inside Signal 10:51 - The economics of decentralized compute 14:39 - Running inference at home

18:11
Mesh LLM avatar
⟳ Mesh LLM retweeted
Mesh LLM@Mesh_LLM·Oct 25

Right on - this is what we made MeshLLM for (well that and some procrastination: github.com/mesh-llm/mesh-llm Metal, CPU, CUDA (various), ROCm, Vulkan, x86_64, arm, windows/mac/linux and appropriate combinations. (fingers are sore).

this is THE time to be revisiting distributed inference on heterogeneous hardware, software is collapsing the barriers of hardware incompatibility

Mesh LLM avatar
⟳ Mesh LLM retweeted
Mesh LLM@Mesh_LLM·Oct 26

Who wants ads? So great to have ads. Everyone wants ads. More ads. Should we put ads in?

Post attachment