←
Box Weights17 posts
Box Weights avatar

Box Weights

@boxweights

Local inference, quants, Hermes, whatever fits this box. Flags, tok/s, what OOM’d.

◷ Joined September 2026▧ 4 Photos and videos
Box Weights avatar
Box Weights@boxweights·Sep 30

Cut down your experimenting time and have more fun testing and optimizing your local box w/ LLama.cpp Server and Hermes Desktop! @ggerganov @Teknium @NousResearch

If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop. This is tailored to my workflow but it can be refactored to suit your needs easily. 🪄 Here's what it does ✨ 👇

1:43
Box Weights avatar
⟳ Box Weights retweeted
Box Weights@boxweights·Sep 30

Cut down your experimenting time and have more fun testing and optimizing your local box! @ggerganov @Teknium @NousResearch

If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop. This is tailored to my workflow but it can be refactored to suit your needs easily. 🪄 Here's what it does ✨ 👇

1:43
Veralidity avatar
⟳ Box Weights retweeted
Veralidity@veralidity·Sep 30

We’re a digital commerce shop, but after hours our Founder & CTO is still pushing boundaries and building the tooling he actually uses. Want to see the raw code, local AI setups, fine-tuning trials, and errors? Follow @boxweights 🔥

Damn this is awesome! I just hit 196 tok/s sustained on my RTX 4090 with Qwen3.8 Flash Next Coder, using a real-world coding task in my @NousResearch Hermes Agent Desktop * 23.9GB VRAM packed with 8501 experts, KV Cache Q8, Threadripper 5955WX + 247GB RAM

Post attachmentPost attachmentPost attachment
Box Weights avatar
Box Weights@boxweights·Sep 30

Cut down your experimenting time and have more fun testing and optimizing your local box! @ggerganov @Teknium @NousResearch @Alibaba_Qwen @googlegemma @NVIDIAGeForce @Supermicro @ubuntu

If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop. This is tailored to my workflow but it can be refactored to suit your needs easily. 🪄 Here's what it does ✨ 👇

1:43
Box Weights avatar
Box Weights@boxweights·Sep 30

If this is something that you would want to use let me know and I'll get it packaged up and available for you to use!

Box Weights avatar
Box Weights@boxweights·Sep 30

✅ Add, Edit, Delete Model Versions ✅ Star a Default Version for each Model ✅ Star a Default Model Version as your Main Model used ✅ Automatically Switch Models when Re/Starting without having to refresh Providers/Models manually ✅ Built-in monitoring, GPU, CPU, RAM, TOKs 👇

Box Weights avatar
Box Weights@boxweights·Sep 30

✅Configure global settings such as llama-server path, models directory, log path, default flags, etc ✅ Create different startup Versions for each model you use. This allows you to test different variations of the same model easily ✅ Start, Stop, Restart each model Version 👇

Box Weights avatar
Box Weights@boxweights·Sep 30

If you're like me an are constantly trying to test and optimize your local box then you'll enjoy my llama-server Manager plugin for @NousResearch #Hermes Desktop. This is tailored to my workflow but it can be refactored to suit your needs easily. 🪄 Here's what it does ✨ 👇

1:43
Box Weights avatar
Box Weights@boxweights·Sep 29

Damn this is awesome! I just hit 196 tok/s sustained on my RTX 4090 with Qwen3.8 Flash Next Coder, using a real-world coding task in my @NousResearch Hermes Agent Desktop * 23.9GB VRAM packed with 8501 experts, KV Cache Q8, Threadripper 5955WX + 247GB RAM

Post attachmentPost attachmentPost attachment

Qwen3.8-Flash-Next first 32K tokens are 4%-10% faster, LOW RAM mode added (for low hardware). New update here github.com/Niko1221/Strata/r…