←
Brian McCrindle1,170 posts
Brian McCrindle avatar

Brian McCrindle

@mccrinbc

Founding Engineer @MacrocosmosAI | @bitstarterAI

◷ Joined January 2020▧ 54 Photos and videos
IOTA ・ SN9 avatar
⟳ Brian McCrindle retweeted
IOTA ・ SN9@IOTA_SN9·Nov 13

There is no global GPU shortage. Only poor coordination.

0:02
Apex・SN1 avatar
⟳ Brian McCrindle retweeted
Apex・SN1@Apex_SN1·Nov 9

Our latest competition solves a practical problem for our friends at @urnetwork + their 350k monthly users: They set a goal: *reduce p95 latency by 20%* End of round 1: *top agent reduced latency by 63%* + 14 independent solutions exceed their stated goal The most impressive part is that the top solution opens a new door to many further innovations. Only 5 rounds left.

Round 1 of our competition on @Apex_SN1 has a winner! Entrants were scored on p95 request time, which tracks the slow end, basically the long waits that make a connection feel stuck In testing, the winning code cut that from 19.6 to 7.1 seconds, a 63% drop. Larger downloads also ran about twice as fast The winning code is now the starting point for round 2 Thanks to everyone who contributed so we can bring these gains to everyone using UR

0:20
Gyles | TAO.com avatar
⟳ Brian McCrindle retweeted
Gyles | TAO.com@Gylez·Nov 6

Token usage at all-time highs. Politicians who can barely open a PDF are holding emergency meetings to slow it down. AI Prohibition is coming. History tells us how that ends. You can’t regulate away demand, and I think some of it finds a home in permissionless protocols.

Pratik Gandhi avatar
⟳ Brian McCrindle retweeted
Pratik Gandhi@pratikg·Nov 5

✅ BENCHMARKED: IOTA Decide 4B v0.4, an open-weight decision model from @MacrocosmosAI, running on my 64GB M5 Pro Mac mini You hand it a state (a ticket, a policy, an agent trace) plus questions and one forward pass returns a probability for every answer. With their bundled loader on the Mac's GPU in full bf16, it loaded in 12.8 seconds and peaked at 10.7 GB. Short ticket, about 130 tokens: - yes/no: 88 ms - 3-way choice: 105 ms - 3-level score: 88 ms That's about 11 decisions a second on a desktop Mac. Their own example, a refund asked for 45 days after purchase on a 30-day policy, answered all 3 questions in 277 ms with the same answers on every repeat: - eligible: 2.2% yes - route: refunds team, 86% - urgency: Low, 63% Longer states take about 0.6 s per decision at 1k tokens and 2.6 to 3.0 s at 4k. These numbers are the floor. On Mac its linear-attention layers still run on PyTorch's generic kernels, so an optimized Metal path should make it faster. Their 33 ms figure is on a dedicated GPU. If you're building agents, about 90 ms a decision is fast enough to check a policy before an agent acts or route tickets on your own machine, with no per-call bill. I didn't benchmark accuracy, but it gave the same answers on their refund example every run.

0:12

Introducing IOTA Decide 4B, our first public model. Pareto optimal. Built different. Trained across globally distributed GPUs with our SDK. Open weights, Apache-2.0. 🤗 huggingface.co/macrocosm-os/…

0:25
Sami Kassab avatar
⟳ Brian McCrindle retweeted
Sami Kassab@Old_Samster·Nov 4

based ai lab - Trained distributed, end to end, using streaming DiLoCo across 16 RTX A4000s. - Training data came from our own stack: we generated and labelled it with distributed batch inference

Introducing IOTA Decide 4B, our first public model. Pareto optimal. Built different. Trained across globally distributed GPUs with our SDK. Open weights, Apache-2.0. 🤗 huggingface.co/macrocosm-os/…

0:25
Openτensor Foundaτion avatar
⟳ Brian McCrindle retweeted
Openτensor Foundaτion@opentensor·Nov 4

Decentralized training on Bittensor is now delivering competitive open models. @MacrocosmosAI just released IOTA Decide 4B, fine-tuned across 16 GPUs around the world and open for anyone to build on.

Introducing IOTA Decide 4B, our first public model. Pareto optimal. Built different. Trained across globally distributed GPUs with our SDK. Open weights, Apache-2.0. 🤗 huggingface.co/macrocosm-os/…

0:25
IOTA ・ SN9 avatar
⟳ Brian McCrindle retweeted
IOTA ・ SN9@IOTA_SN9·Nov 4

We've always believed that distributed technologies can be indistinguishable from their centralized counterparts. Our first public model shows that they can be pareto optimal too. Introducing IOTA Decide 4B.

Introducing IOTA Decide 4B, our first public model. Pareto optimal. Built different. Trained across globally distributed GPUs with our SDK. Open weights, Apache-2.0. 🤗 huggingface.co/macrocosm-os/…

0:25
Brian McCrindle avatar
Brian McCrindle@mccrinbc·Nov 4

Our first open model from @MacrocosmosAI, trained entirely distributed, Pareto-optimal on Jev-Bench! 🤗 huggingface.co/macrocosm-os/…

IOTA Decide 4B sits on the Pareto frontier: no smaller model scores higher. It's cheap to run. ~$0.0037 per 1,000 decisions self-hosted.

Post attachment
Macrocosmos avatar
⟳ Brian McCrindle retweeted
Macrocosmos@MacrocosmosAI·Nov 4

Introducing IOTA Decide 4B, our first public model. Pareto optimal. Built different. Trained across globally distributed GPUs with our SDK. Open weights, Apache-2.0. 🤗 huggingface.co/macrocosm-os/…

0:25
IOTA ・ SN9 avatar
⟳ Brian McCrindle retweeted
IOTA ・ SN9@IOTA_SN9·Nov 3

Nobody should care whether a model was trained distributed. They should care that it’s useful. We’ve been using models we trained across distributed GPUs, and they’re very good. Sharing one soon.

Glenn Greenwald avatar
⟳ Brian McCrindle retweeted
Glenn Greenwald@ggreenwald·Oct 30

Free speech in the UK:

Anyone planning to attend a demonstration in support of Palestine Action in the vicinity of the US Embassy on Saturday, 31 October will face arrest as part of a robust policing operation. Expressing support for the group is an offence under the Terrorism Act following its proscription in June 2025, a decision upheld by the Court of Appeal in June 2026. Read more: news.met.police.uk/news/met-…

Will Squires avatar
⟳ Brian McCrindle retweeted
Will Squires@WSquires·Oct 30

Local inference isn't a terrible allocation of resources if you have technology that can unite distributed, heterogenous, and idle (re, off 95%) of the time compute to do frontier scale workloads. That's the core of our thesis - and more broadly that maximising utilisation is the core next step in the AI race.

sam: local inference is a terrible allocation of scarce resources “a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment somehow many people's intuition is the opposite of that”

0:45