←
Cactus Compute163 posts
Cactus Compute avatar

Cactus Compute

@cactuscompute

Foundation Models For Tiny Devices

◷ Joined October 2025▧ 43 Photos and videos
Cactus Compute avatar
Cactus Compute@cactuscompute·Sep 17

We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4 Flash. One set of weights, every depth from 2 to 20 layers a model of its own, 25-121M parameters at CQ2-bit, built on our Simple Attention Networks and running locally at up to 4k tokens/sec decode speed on a Raspberry Pi 5. Needle does not chat. Every turn is a function call: give it the tools your app exposes and it picks the right ones and fills every argument from what the user said, or hand it a schema and it returns a typed record. Ask for something no tool covers and you get an empty list, not a guess. That trade is lets 121M parameters trained on 360B tokens of structured data beat models 10x their size on mobile tool calls and match 2-3x bigger models on structured JSON extraction. It runs on mobiles, wearables, smart home devices, small robots and microcontrollers, with prebuilt engines for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, the browser and WASI hosts. Try it in your browser: cactuscompute.com/needle

0:51
Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 12

3) In one test, 3-bit code returned 11 where the limit was 10. We copied one internal calculation result from the original model. The weights stayed compressed; the code then passed its tests. Fixing an incorrect step with a more capable model heals the trajectory, making hybrid inference between a weak and more capable model inspired (speculative decode, edge-cloud hybrid, etc.)

Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 12

2) The ability to fix its own mistakes matter and accelerates degradation over time.

Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 12

Setup: - Model: Qwen3-4B at 4-bit vs 3-bit, both MLX's quants. - Benchmark: 32 eight-task coding sequences from 12 templates; shared history, greedy decoding, test feedback, and up to 3 attempts per task. Findings: 1) Small per-task loss compounds really fast over multiple tasks.

Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 12

We compressed Nemotron-3.5-SuperVL-120B to 1.418 effective bits per weight using Cactus Quants, the algorithm behind our foundation models Needle, Whistle and Mirage. “1-bit” labels can describe weight codes alone. Our 1.418 bits/weight is calculated from the entire file. The complete 124.45B-parameter checkpoint fits in 22.06 GB, including the vision tower, projector and MTP head. On a 36GB M5 Max MacBook Pro, we measured approximately 47 tokens/sec with a 512-token prompt and 128 output tokens, using verified MTP. That’s median warmed, batch-one decode throughput across three runs, with 21.0 GiB peak process memory footprint. Across 18 reported benchmarks, the mean relative score reduction is 3.6%. DeepSWE is reported separately. BF16 solves 24 tasks and CQ seven, a substantial weakness on long agentic coding runs. The model card reports 113 tasks; a saved CQ aggregate records 110. Our analysis explains that unresolved discrepancy. Weights, detailed evaluations and a getting-started guide for Cactus MLX with custom Metal kernels are in the HuggingFace repo. Cactus code is Apache 2.0: huggingface.co/Cactus-Comput… We investigated possible failure mechanisms with short, controlled coding experiments. The five posts below separate what we measured from what remains unknown about DeepSWE.

0:43
Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 6

Our research at Cactus aims to compress intelligence to its smallest pragmatic form for every day consumer hardware. We share early findings to workshops, like our 5/5 NeurIps 2026 acceptance. But we also publish other techniques on our blog, such as the early research that birthed Mirage. 1. Structured Visuo-Semantic Perception: cactuscompute.com/blog/struc… 2. HConvGLU for Channel Mixing in Image Encoders: cactuscompute.com/blog/hconv… 3. Instance Segmentation from Shared Features: cactuscompute.com/blog/insta… 4. Gabor Initialisation with Matched Filter Energy: cactuscompute.com/blog/gabor… 5. CTC Decoding Across Image Windows: cactuscompute.com/blog/ocr-d… Cactus technologies (Needle, Whistle, Cactus Quants, Cactus Hybrid and Engine) are now used by 100k developers monthly. Thanks a million for the support, cheers!

Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 6

Mirage reasons over images, producing a compact yet descriptive response. You can inspect what the model describes and infers alongside the image.

Post attachment
Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 6

Mirage extracts relation between object IDs, which link back to their boxes and masks. Your app can follow those connections directly in the result.

Post attachment
Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 6

Mirage returns a mask for each described object. Use polygon outlines to draw a contour or bitmap masks to select pixels.

Post attachment
Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 6

Mirage's structured responses includes objects with their bounding boxes, detection score and properties such as color or shape.

Post attachment
Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 6

Unlike VLMs that generate paragraphs and leave the the information parsing to developers, Mirage describes images in a compact structured format.

Post attachment
Cactus Compute avatar
Cactus Compute@cactuscompute·Nov 6

Today we release Mirage: A 22 MB visual-language model that runs scene descriptions + object detection + OCR + segmentation + spatial relations analysis + extractions + decisions, in a single pass. It can do all that in 74 ms on Samsung Galaxy S26 NPU or 107 ms on its CPU. Mirage has a 7.8M encoder + a 40.8M decoder at 2bits, and describes images in English, German, French, Spanish, Italian, Dutch and Polish. We trade general language capacity, to compete top small VLMs today (SmolVLM2-500M, LFM2.5-450M and Florence2-230m) on these tasks. On 256 VQAv2 questions, feeding Mirage's structured output and full trace to text-only Qwen3-0.6B raised its score from 9.3 to 31.3 out of 100, without fine-tuning either model. Read more and benchmark it yourself here: cactuscompute.com/blog/mirag…

Post attachment