←
Qubitium7,978 posts
Qubitium avatar

Qubitium

@qubitium

GPT-QModel, QvQ ultra low bit quants. Contributor to stuff you are probably using like sglang, vllm, transformers, rocm. ucla alum

◷ Joined February 2020▧ 932 Photos and videos
Qubitium avatar
Qubitium@qubitium·Nov 17

GPT-QModel v7.11 now on pypi! 👇 🧠 MACaQ, NF12 🛠 Tenstorrent HW, bf8_b, bf4_b (blockfloat) 📀 Lower vram usage for many pipelines 🔃 Major EXL3/AWQ updates including accuracy and quantization performance 💾 ROCm and MLX compatible kernels

Post attachment
Qubitium avatar
Qubitium@qubitium·9h

GPT-QModel 7.11.4 is out. 👇 * Less calibration memory for AWQ + Paro * Accuracy fixes for SmoothMSE, QQQ + AWQ * Faster large-batch Marlin prefill * Compiled generation for Torch QQQ/VPTQ * Fixed Transformers auto CPU/disk offload github.com/ModelCloud/GPTQMo…

Post attachment
Qubitium avatar
Qubitium@qubitium·11h

Is it floating or anchored?

Selling Oculus to Facebook in 2014 gave me money for helicopter flight lessons and developing a seastead called SLAB with a 1,000 year design life. The goal was to be cheaper than land and absent state/local laws that cripple critical American industries like metal processing, medical research/treatment, desalination, oil refining, chemical engineering, and more. Me and a few close friends cashed it up at a $1B valuation back in 2017, and while it still has a long way to go, there has been significant progress towards my goal of reviving manifest destiny and westward expansion. At a time when most of the seasteading community wanted to escape the US and smoke pot, this was the one project explicitly designed to make America stronger, a relief valve for ill considered or outdated rules and regulations. I am only talking about it now because a few media outlets got wind of me and some friends topping up the coffers and plan on calling SLAB my new floating AI datacenter project. It is way cooler than that, way older than that, and I hate looking like a hype-chasing consensus loser, so now you have to read this tweet.

Post attachment
Qubitium avatar
Qubitium@qubitium·21h

GPT-QModel 7.11.2 is out.👇 • CPU: full-graph torch.compile on supported kernels • Less VRAM for GPTQ/EXL3 quantization • NF12/MACaQ checkpoint resume fixed • New: SenseNova-Loop 9B + Decision Transformer • GGUF: ggml 1.0 + FP8 E4M3 github.com/ModelCloud/GPTQMo…

Post attachment
Qubitium avatar
Qubitium@qubitium·Nov 17

👀

#SenseNova-Loop-9B is officially here — now #OpenSource! Just 9B parameters, competitive with Qwen3.8-27B across multiple benchmarks. 🔹 Big gains over Preview: NL2Repo 19.91 → 41.36; AutomationBench 14.33 → 36.67; GAIA 37.86 → 69.90 🔹 Loop + mHC: 32 unique layers, 56 layer executions 🔹 Day-0 4-bit AWQ weights (~4.9GB), with VRAM needs as low as ~9 GiB at 64K context 🔹 Up to 512K context, with text and image inputs See what a 9B model can build: animated posters, interactive websites and 3D scenes. 💻 GitHub: github.com/OpenSenseNova/Sen… 🤗 HF: huggingface.co/collections/s… 👾 Discord: discord.com/invite/BuTXPHmQu…

Post attachment
Qubitium avatar
Qubitium@qubitium·Nov 17

wtf

Did you know that running your Mac for more than 50 days without a reboot will fuck with your internet connection? Yes, really. There is a 32-bit integer timing your TCP connections and if you let it overflow MacOS will run into all kinds of fucky issues with its TCP stack. For example I was unable to upload things bigger than 4MB. I was wondering why my Claude sessions started to hang seemingly at random. The fix? Literally just reboot your machine. That's it. What the fuck photon.codes/blog/we-found-a…

Post attachment
Qubitium avatar
Qubitium@qubitium·Nov 17

GPT-QModel's api is morphing at a fast pace. If you need the most up-to-date spec plus doc, visit or point your agent to the repo wiki which is updated multiple times per day in sync with the main branch. github.com/ModelCloud/GPTQMo…

Qubitium avatar
Qubitium@qubitium·Nov 17

Welcome to 2026. Daleks are coming.

This is a big one. We found malicious JavaScript being served through Optimizely to some of the world's largest websites. 178 organisations affected, with everything from crypto theft and card skimming to iOS exploits. Zoom, BBC, CNN, HSBC, Salesforce, GitLab, KLM, IKEA, Equifax, Ledger. The list goes on... Full list in the post. blog.report-uri.com/worlds-l…

Qubitium avatar
Qubitium@qubitium·Nov 17

🛡 Not just shiny new kernels. New quantization support: OLMo Hybrid, Granite SWA / MoE SWA, Dots3 Note. Plus dense/MoE + multimodal fixes, safer checkpoint handling, and runtime checks for vulnerable PyTorch versions.

Qubitium avatar
Qubitium@qubitium·Nov 17

🎛 More control, fewer mystery switches. Typed backend configs unify selection + tuning, with explicit numerical and worker controls. Calibration gets broader dataset inputs, document-isolated AWQ replay, and opt-in attention-sink exclusion.

Qubitium avatar
Qubitium@qubitium·Nov 17

🏎 Less overhead. Keep the weights packed. Opt-in attention + MLP projection fusion, faster quantized embedding lookups, and better small-batch inference. Less packed-weight duplication and shorter-lived scratch. Native backend selection stays intact.

Qubitium avatar
Qubitium@qubitium·Nov 17

🔁 Big quantization runs deserve a resume button. Distributed EXL3 now supports resumable jobs and projection-level recovery. Plus broader mixed-bit allocation across AWQ / EXL3 / GGUF, and SmoothQuant preprocessing.

Qubitium avatar
Qubitium@qubitium·Nov 17

🛠 Tenstorrent joins the inference stack. Packed GPTQ via Torch/XLA + native TTNN, multi-card execution, and Llama serving with paged KV + chunked prefill. Compatible block-float checkpoints also run on CUDA, ROCm and CPU.

Qubitium avatar
Qubitium@qubitium·Nov 17

🧊 NF12: compress the weights, not the truth. Eligible BF16 weights go from 16 to 12 bits during calibration—with exact restoration before quantization. Packed inference now adds native Metal alongside CUDA, HIP and Torch.

Qubitium avatar
Qubitium@qubitium·Nov 17

⚡ Meet MACaQ: native MXFP4 W4A4 quantization. 4-bit weights AND activations, with packed checkpoints and inference on CUDA, ROCm and Apple silicon. Need a portable path? There's a Torch fallback, too. github.com/ModelCloud/GPTQMo…