role-model is here: a model router for Pi that works with (almost) any cloud model as well as local models.
Set up your pool of models, whether GPT 5.5 and DeepSeek V4, or Qwen 27B and Qwen 35B running locally, or combine and route between your local model for general chat and tool use and cloud models for more difficult tasks.
role-model is a router runtime and and a forward-looking routing protocol that can enable consumer applications like Pi to classify its requests with required capabilities, task-type and suitable roles for better matching between task, role, and model.
pi-role-model is a package for Pi that lets Pi manage and interact with role-model. role-model works perfectly with Pi even without the package - just configure your models in role-model, then configure role-model as your provider in Pi and choose routing alias.
Install the package for Pi:
’pi install npm:@try-works/pi-role-model’
Then let your agent install and configure the role-model runtime or point it to:
‘github.com/try-works/role-mo…’
role-model is in an early alpha release - let me know bugs, issues or usability questions. also, tell me if it just happens to work perfectly for you!
there's lots more to do and you can help by installing and running role-model, and by reposting this tweet!
thx
Alright let's talk about how we're building @uisdk to be agent-first.
UI SDK has first-class WebMCP support. UI built with the SDK comes with its own tools. Agents read your components and call their actions directly. No screenshots. No clicking around.
In our benchmark: ~50% fewer tool calls, ~75% fewer tokens and ~70% cheaper than existing libraries and tools.
See the numbers, and how to run the benchmark yourself at
ui-sdk.dev/docs/agents#bench…
According to multiple sources, Xiaomi’s large language model (LLM) unit, MiMo, is at a critical juncture over a potential spin-off. Several VC firms and CVC investors have initiated preliminary discussions, but Xiaomi has yet to formulate a concrete spin-off and fundraising plan.
Sources indicate that Xiaomi founder Lei Jun and MiMo head Luo Fuli disagree over the proposed spin-off, with limited progress in discussions. Luo has reportedly approached a small number of investors and financial advisors (FAs) and may consider launching her own startup.
mp.weixin.qq.com/s/7LwGupEtm…
We’ve heard again and again that enterprises don’t want to pour all their valuable IP, data and tacit knowledge into someone else’s model. They want control of their own weights.
*Co-residence is the ability to compose independently trained fine-tunes into one model, without capability regression from adapter interference." Is fucking WIZARDY
Part 1: We now enter the era of hyper-composability.
For the first time, it is economically feasible to serve individually fine tuned models at scale.
And today, we open source it all.
We can now serve heterogeneous batches of custom fine-tunes, without incurring LoRA’s per matrix bandwidth overhead. This allows us to economically serve this in large batches without incurring penalties for co-residing with default model users in the same batch.
With Hypertune, we solve co-residence and are able to train co-resident adapters.
Co-residence is the ability to compose independently trained fine-tunes into one model, without capability regression from adapter interference.
Hypertune further matches LoRA in RL performance across 13 domains with 100x-500x fewer parameters.
Serving heterogenous LoRA fine-tunes in one batch is prohibitively expensive. Each fine-tune variant needs its own low-rank adapter at every matrix and is only used for a few tokens, leading to many narrow matmuls and poor utilization.
This is exacerbated for MoEs, where an adapter only engages when the token simultaneously selects a certain fine-tune and expert. This combinatorial blowup kills any remaining prospects of fast inference for fine-tuned / custom models.
Hypertune introduces soft prefill, a trained KV adapter. Soft prefill can be trained using GRPO, SFT or any other first order method.
As a consequence, Hypertune circumvents the LoRA overhead because soft prefills live entirely in the KV cache. This allows us to easily integrate it into existing serving stacks by prepending the soft prefills into paged KV blocks.
Note: Context compression using cartridges requires copious amounts of synthetically generated self-study data for context distillation. Hypertune, instead, accommodates a panoply of gradient based methods for a wider range of objectives.
Caveat: Comparisons with full-parameter GRPO are not required, because under RL, LoRA already matches it. LoRA Without Regret by @thinkymachines finds that LoRA performs equivalently to full fine-tuning for RL even at very small ranks, down to rank 1.
The takeaway:
Our gains indicate that substantial amounts of in-context learning capacity is left on the table by discrete prompts.
This could be either because soft prompts reach conditioning states that no token sequence can express, or because their continuous parameterization makes good conditioning states far easier to find via gradient descent (even if a discrete equivalent exists).
Companies and individuals can finally fine tune models on their private data, making models hyper-optimized to their needs, all without dramatically overpaying (75% cheaper than LoRA for inference).
Here's the open sourced repo: github.com/Hyperstitioncc/Hy…
Part 2 coming out later this week.
Pushing Qwen Next Flash on my Mac Studio M5 Ultra (256GB) 🚀
🔥 17 concurrent requests (16 actively decoding)
⚡ 453 tokens/sec median aggregate decode
📈 ~438 tokens/sec sustained right now
That's roughly 27–28 tok/s per request, with all 16 decoding in parallel.
And the interesting part? This isn't a synthetic benchmark. It's a real workload, running continuously.
Apple Silicon + MoE models are a pretty fascinating combination. 🍎
Introducing iOS for the web.
This is Telegram (written in Swift and Objective-C), running fully in a browser on our custom renderer. No emulator, No iPhone required.
You can infinitely branch the entire codebase by copy pasting a frame and make changes to each frame.
We built a compiler+renderer (in Rust) that allows you to easily design on top of a native iOS codebase. Most edits land in under a second and is much much faster then xcode builds.
Our goal is to take any codebase and turn it into a design environment that fully runs in your browser!
That is huge! Once we get to the cross-model-family KV transfer, that’s gonna be revolutionary. Imagine doing prefill on qwen 27B and decode on OOM bigger model, >1T.
This is also where heterogeneous compute will play a huge role.
Today we’re launching cross-model KV cache transfer from @OneTriangle, a step towards efficient model communication.
Up to 10.8x faster TTFT at 1M context. Prefill small model, transfer KV cache, decode on large. SDK live for Qwen3.5-4B → Qwen3.8-27B, beta open for more model pairs.
Never ask a man his salary,
a woman her age
or an inference provider what happens to their business if token prices get cut in half over the next 6 months
i posted this a month ago and we now have 1000 GPUs
40% GB300s and 60% B300s in hand
all on short term contracts
everyone saying gpu shortage, impossible without 5 year commits, credit is dead blah blah
everyone just repeating what someone else told them
Introducing Fluid Functionalism Foldkit
A port of @micka_design's Fluid Functionalism to @foldkit & Foldkit UI
The goal was to be faithful to the source without reimplementing the minutia of the underlying component libraries
GitHub: github.com/timhanlon/fluid-f…
Demo: fff.timhanlon.com/
Likely some rough edges, and the newer components like data table are still WIP