←
Aleph Alpha637 posts
Aleph Alpha avatar✓

Aleph Alpha ✓

@Aleph__Alpha

Our Specialized Sovereign Large Language Model for Mission Critical Environments is here: aleph-alpha.com/en/kolibri/

◷ Joined December 2018▧ 268 Photos and videos
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Oct 3

Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.

0:04
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Nov 3

Does it work? On the SFT checkpoint, reasoning length orders low < medium < high on 23 of 24 benchmarks. At a fixed effort, hard prompts still get 32.5 times the reasoning of easy ones. After RL, the released model keeps 98% of its high score at low, with 76% of the tokens.

0:09
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Nov 3

A hard cut at the thresholds would teach the model a token count to stop at. So the label is drawn softly around them, and each group still ends up close to 30% low, 40% medium, 30% high.

0:10
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Nov 3

Inside one dataset, difficulty still spreads the reasoning length. We used an LLM judge distilled into a 2B grader to rate every prompt either easy, medium, or hard. Each of the 176 (dataset, difficulty) groups then gets its own thresholds.

0:06
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Nov 3

So every dataset gets its own 30th and 70th percentile, as a cheap stand-in for its domain. Across 74 datasets the median trace runs from 27 to 90K tokens.

0:09
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Nov 3

A simple baseline is one pair of length thresholds to bucket each sample into one of three reasoning efforts. It fails because reasoning length follows the domain: 49% of chat would be low and 83% of math high. Drag the thresholds anywhere and some domain stays lopsided.

0:17
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Nov 3

Kolibri can reason at four efforts: none, low, medium, and high. Our SFT training traces didn't have effort labels, so we had to label them post-hoc. A new blog post by @L_Salewski explains how we did this, with interactive figures 🧵

0:08
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Oct 29

Knowing where every element sits on the page makes the model useful beyond extraction. For example, for source validation in chat assistants. As Parsy keeps the bounding box of every element, an answer can point to the exact region on the page it came from. Read the full post by @ABCalec, @yassinekh67, and Lukas Tatge: aleph-alpha.com/en/blog/pars…

Post attachment
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Oct 29

We re-ran ParseBench with the public scoring code, and the open-weight models matched the leaderboard while LlamaParse initially did not. @llama_index traced the gap and refreshed the scores the same day, and is adding scheduled runs so anyone can reproduce the leaderboard.

Post attachment
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Oct 29

In the sixth round of training and label cleaning, the model passed the harness. Our explanation for the model overtaking the harness: the latter got its quality at run time by combining models and catching their mistakes on every page. The model instead learns from the cleaned result of that process.

Post attachment
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Oct 29

Parsy is currently the best parsing model in the world. Read on for how we built this model on top of our parsing harness. 🧵

Post attachment
Aleph Alpha avatar
Aleph Alpha@Aleph__Alpha·Oct 27

Foreign cultural context is the harder problem. Questions about another country’s laws still test that country’s culture, but in German. A translation often implicitly decides what that cultural context is, influencing what that eval measures.