←
netrunner10,376 posts
netrunner avatar

netrunner

@plotarmordev

Cofounder & engineer. Running local AI, building things with it, and sharing what I learn.

◷ Joined April 2022▧ 551 Photos and videos
netrunner avatar
netrunner@plotarmordev·45m

I run some coding agents on a local model so my code never leaves my Mac. One old API key in a config and it can use a cloud model instead. While it works I run: lsof -nP -iTCP -sTCP:ESTABLISHED Any address that isn't my Mac or Spark means it's using the cloud.

netrunner avatar
netrunner@plotarmordev·2h

Labs should put the download size in GB next to the parameter count... Tencent says Hy4 Flash is 224B. Its Hugging Face page says 155.5B. Neither tells me if it fits my Sparks. The files do: 440 GB in BF16, 287 GB in FP8. Is it too hard?

netrunner avatar
netrunner@plotarmordev·4h

Waiting is cheap on hardware you own. A rented GPU bills by the hour while your agent sits idle. My Mac keeps its model in memory and the wait costs me just a little wall power.

netrunner avatar
netrunner@plotarmordev·5h

Getting 2 GB back on a headless Spark is one click now! @MiaAI_lab added spark-attic to sparkDash. It frees the memory the Spark keeps for a monitor, even when none is plugged in. I flagged that reserve in October.

‼️ Your DGX Spark might be saving RAM for a monitor that you don't use. If it runs headless, set Display Reserved Memory to Auto in UEFI. With no display connected it keeps 0.5 GB instead of 2 GB. Steps below 🧵

Post attachment
netrunner avatar
netrunner@plotarmordev·7h

Hy4 Flash is open now, it's 287 GB in FP8. My NVFP4 estimate is about 220 GB. Not half, because only 153B of its weights get quantized. So NVFP4 means TP=2 on Sparks, with little room left for context btw.

Post attachment

Hy4 flash is now open-weight, including Hy4 flash-FP8 and Hy4 flash-Base. 224B (12B active), 1M context, natively multimodal. Fast, efficient and built for everyday productivity. It's yours now. And stay tuned for Hy4 coming up next! 🤗: huggingface.co/tencent/Hy4-f… 🐙: github.com/Tencent-Hunyuan/H… 📄: hy.tencent.ai/research/hy4-f… Released under the Tencent Hy License. See full terms here: huggingface.co/tencent/Hy4-f…

Post attachment
netrunner avatar
netrunner@plotarmordev·8h

Look at this, my local agent thought "OMP" meant 1Password (????). It stopped and asked, because I limited what it could touch on my Mac. Now imagine a weaker model with full access, set up by someone new to this... Does this make you understand the new warning on MacOS?

Post attachmentPost attachment
netrunner avatar
netrunner@plotarmordev·11h

Run a few agents on one Spark and tok/s isn't what you notice. It's how long the last request in line waits for its first token, and that's down to scheduling, not decode speed.

netrunner avatar
netrunner@plotarmordev·13h

Unsloth shipped computer use for local models, in beta. @danielhanchen's demo shows Qwen3.8 27B turning receipts into an expense report and editing photos in GIMP, plus Blender and Unreal Engine. I already have other solutions for it but this is worth a try.

Unsloth now has computer use for any local model! It's still beta but we had Qwen3.8-27B make an expense report from receipts & used GIMP to scale photos, add captions & export it! Also we added driving Unreal Engine, Blender & more to local models! github.com/unslothai/unsloth…

netrunner avatar
netrunner@plotarmordev·16h

Wow @GoogleDeepMind’s Carbon looks good, this might have convinced me to give it a try! Good job Google

Post attachment

I will not be writing a full review of GPT-6.2 Astra / Sol. I had a terrible experience testing them. They were essentially unusable. I do not recommend these models.

netrunner avatar
netrunner@plotarmordev·21h

What stays saved on my drives: 1) Every quant I run, not just base weights 2) The engine build for each 3) Chat templates and tokenizers 4) My launch flags 5) Voice, embeddings and rerankers 6) All my sessions Weights you can always download again, can't say the same for a working setup

netrunner avatar
netrunner@plotarmordev·23h

On the Spark time, 80.6s is TensorFold's Spark port, which is newer than the Mac one. It's not the Spark's ceiling. The Mac build has days of tuning behind it, and the 12.6s demonstrate that on full BF16 weights. Spark rematch once the port catches up!

netrunner avatar
netrunner@plotarmordev·23h

Ok I’ll take it, small chance Jensen comes back as CEO and saves the future of open weight

Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen

netrunner avatar
netrunner@plotarmordev·Nov 17

Tencent's Hy4 Flash is API only but $0.07 in/$0.281 out per 1M tokens, half that on @openrouter for now. Tencent says it matches Haiku 5.5 on daily work.

Post attachment
netrunner avatar
netrunner@plotarmordev·Nov 17

Confused with Codex. On the $100 plan, Daybreak is the model I actually like, not Astra. And I burn through the limit in no time. I don't want to go back to $200, but GLM, Kimi and Claude run out too. Local covers a lot, not everything. What would you do?