←
Graphcore Research190 posts
Graphcore Research avatar

Graphcore Research

@GCResearchTeam

Our mission is to follow and contribute to the advancement of AI research, aiming to characterise the computational requirements of machine intelligence.

◷ Joined January 2024▧ 32 Photos and videos
Graphcore Research avatar
Graphcore Research@GCResearchTeam·Aug 27

11 billion parameters. On a phone. We wanted to see whether Llama-3.2-11B-Vision-Instruct — far too large for a phone in its original form — could fit into a ~4 GB budget. The answer: yes. Here's how 🧵

Graphcore Research avatar
Graphcore Research@GCResearchTeam·Aug 27

The data recipe mattered more than we expected. Generic prompt sampling, mixed prompt formats and varied answer lengths were all important. With the wrong synthetic data, training loss can improve while downstream performance gets worse. Fun 😅

Graphcore Research avatar
Graphcore Research@GCResearchTeam·Aug 27

Format design was only half the battle. Directly casting below ~3.5 bits/parameter hurts task quality rapidly, so we use quantisation-aware training + self-distillation: the original model teaches its quantised student on synthetic image-text data.

Graphcore Research avatar
Graphcore Research@GCResearchTeam·Aug 27

And it isn’t just small on disk. On a Pixel 8a, our custom C++ implementation can run the 3.73 GB model at ~3.8 tokens/s during autoregressive generation. With INT8 weights, the model doesn't fit in memory.

Graphcore Research avatar
Graphcore Research@GCResearchTeam·Aug 27

Why vector quantisation? Below 3 bits/parameter, reconstruction error gets brutal. S3D8 uses short 3-value centroids + separate sign bits, improving the rate/distortion trade-off while keeping decoding friendly to Arm register lookup tables.

Post attachment
Graphcore Research avatar
Graphcore Research@GCResearchTeam·Aug 27

The key is S3D8, our new 2.7-bit weight format for Arm CPUs. It packs 3 weights into 8 bits, then decodes them to channel-scaled INT8 so we can use integer matrix-multiply instructions already present in modern mobile CPUs.

Graphcore Research avatar
Graphcore Research@GCResearchTeam·May 18

A tiny implementation detail in low-precision arithmetic could be biasing your AI training 😲 This interactive deep dive from Graphcore Research's @Awfidius uncovers a subtle failure mode in stochastic rounding that only appears when randomness is limited, and how it can be addressed with one simple fix 🎲 Check it out in the link below! 👇

Graphcore Research avatar
Graphcore Research@GCResearchTeam·Mar 12

Would you rather use 1 million × 16-bit weights, 4 million × 4-bit weights, or even 16 million × 1-bit weights? In joint work between Aleph Alpha Research and Graphcore, we asked this question of LLMs — the answer encouraged us to embrace the wonder ✨ of 1-bit weights, which can outperform 4-bit and 16-bit weights on a fixed weight memory budget. In our work - ⚖️ A scaling laws evaluation prompts us to consider very low-bit formats - 📈 Scaled-up tests show the power of memory-matched models with 1-bit weights - ⚡ Kernel benchmarking demonstrates their feasibility for autoregressive inference Read all about it in our blog and paper (link below! ⬇️) Massive thanks to our collaborators at Aleph Alpha Research! Authors: @SohirMaskey, Constantin Eichenberg, @atomicflndr and @douglasahorr

Post attachment
Graphcore Research avatar
Graphcore Research@GCResearchTeam·11 Dec 2025

Our picks for November’s Papers of the Month are here. Out of our shortlisted papers, we spotlight three looking at LLM efficiency from different angles! 📊 First up, How to Scale Second-Order Optimization is studying optimal tuning of second order optimizers such as Muon. 🌫️ In Intelligence per Watt, the authors discuss our favorite metric on large language models: energy efficiency. And how to take advantage of edge AI inference. 🕸️ Finally, Int vs FP: A comprehensive study of fine-grained formats is contributing to an old-timer topic in quantization: integer vs floating (block) point formats. Check out our summaries 👇

Post attachmentPost attachmentPost attachment
Graphcore Research avatar
Graphcore Research@GCResearchTeam·4 Dec 2025

At #NeurIPS in San Diego? Come swing by poster #1016 this afternoon to see Graphcore Researcher Johanna Vielhaben presenting "Beyond Scalars: Concept-Based Alignment Analysis in Vision Transformers"

Post attachment