←
Shaltiel171 posts
Shaltiel avatar

Shaltiel

@SShmidman
◷ Joined December 2013▧ 20 Photos and videos
Rylo avatar
⟳ Shaltiel retweeted
Rylo@RyloHQ·Oct 21

Rylo is partnering with @nvidia to train sign language translation models with supercomputing. Rylo's AI Research team (Rylo Sign) is building the foundation layer for understanding sign language and translating it in real time. The Deaf community has waited long enough for tools that take sign language seriously. We're building them. Full announcement and more info in the thread 🧵

Post attachment
Rylo avatar
⟳ Shaltiel retweeted
Rylo@RyloHQ·Jul 30

Today, we're celebrating our latest funding round at the @NYSE! Our recently announced $85M Series B will ensure we reach more people and remove more of the communication barriers that should have never existed in the first place. Join us in the race toward accessible communication!

Post attachment
Rylo avatar
⟳ Shaltiel retweeted
Rylo@RyloHQ·Jul 20

Introducing: Life [with captions]

0:30
Shaltiel avatar
Shaltiel@SShmidman·3 Dec 2025

We've come a long way from the initial announcement of the Sovereign LLM project - excited to showcase our results. We've written up a tutorial sharing how you can train your own Sovereign LLM using Nvidia Nemotron, NeMo Framework, and DGX Cloud Lepton: lnkd.in/dgCZHCPX Great work together with @nvidia

Post attachment

Beyond excited to announce the official release of: 🚀 Dicta-LM 3.0: Advancing The Frontier of Hebrew Sovereign LLMs 🔥 Dicta-LM 3.0 is a powerful open-weight collection of LLMs with full Hebrew support. View the full announcement here: dicta.org.il/dicta-lm-3 🧵

Shaltiel avatar
Shaltiel@SShmidman·2 Dec 2025

The models were trained on a cluster of over 100 H200 GPUs, on NVIDIA DGX Cloud Lepton. All training was done using the NVIDIA NeMo Framework and the NVIDIA NeMo-RL library. We are extremely grateful to NVIDIA and their technical teams, who made this all possible!

Shaltiel avatar
Shaltiel@SShmidman·2 Dec 2025

Amazingly, the model also performs well on English benchmarks! We evaluated the model using Olmes from @allenai, on the Olmo-3 evaluation set* *See the report for more details

Post attachment
Shaltiel avatar
Shaltiel@SShmidman·2 Dec 2025

💬 We release chat variants of the model, including a 24B thinking model, the first open-source reasoning model to reason *in Hebrew*! The model showcases excellent performance on the Hebrew Chat LLM Leaderboard:

Post attachment
Shaltiel avatar
Shaltiel@SShmidman·2 Dec 2025

🔑 Available in 3 sizes: 24B, 12B, and 1.7B, initialized from powerful base models - Mistral-Small 3.1, Nvidia Nemotron Nano V2, and Qwen3 - and then each model was continuously trained on over 120B tokens of Hebrew text. The models are all available at 🤗 (link at the end) \

Shaltiel avatar
Shaltiel@SShmidman·2 Dec 2025

Beyond excited to announce the official release of: 🚀 Dicta-LM 3.0: Advancing The Frontier of Hebrew Sovereign LLMs 🔥 Dicta-LM 3.0 is a powerful open-weight collection of LLMs with full Hebrew support. View the full announcement here: dicta.org.il/dicta-lm-3 🧵

DAIR.AI avatar
⟳ Shaltiel retweeted
DAIR.AI@dair_ai·30 Nov 2025

7. Training LLMs with Reasoning Traces Ivestigates how reasoning traces from frontier models like DeepSeek-R1 and GPT-OSS can improve smaller language models through post-training.

elvis@omarsar026 Nov 2025

Reasoning models are expensive. Not because the models are huge. It's because they generate thousands of tokens just to think. But what if smaller models could learn to reason efficiently? This new paper compares training 12B models on reasoning traces from two frontier systems: - DeepSeek-R1 - gpt-oss (OpenAI's open-source reasoner) The key finding: gpt-oss traces produce 4x more efficient reasoning. DeepSeek-R1 averages ~15,500 tokens per response. gpt-oss averages ~3,500 tokens. Yet accuracy stays nearly identical across benchmarks. Verbose reasoning doesn't mean better reasoning. Why does this matter? Inference cost scales linearly with tokens. If your reasoning model generates 4x fewer tokens with the same accuracy, you cut costs by 75%. That's a massive efficiency gain. Interesting observation: Nemotron base models already had DeepSeek-R1 traces in pretraining. Training loss on DeepSeek traces started low and stayed flat. Training loss on gpt-oss traces started high and dropped gradually. They showed that the model was learning something new, which also means you can distill reasoning capabilities from frontier models into smaller systems. But the source matters. Different reasoning styles produce different efficiency profiles. (bookmark it) Paper: arxiv. org/abs/2511.19333

Post attachment
elvis avatar
⟳ Shaltiel retweeted
elvis@omarsar0·26 Nov 2025

Reasoning models are expensive. Not because the models are huge. It's because they generate thousands of tokens just to think. But what if smaller models could learn to reason efficiently? This new paper compares training 12B models on reasoning traces from two frontier systems: - DeepSeek-R1 - gpt-oss (OpenAI's open-source reasoner) The key finding: gpt-oss traces produce 4x more efficient reasoning. DeepSeek-R1 averages ~15,500 tokens per response. gpt-oss averages ~3,500 tokens. Yet accuracy stays nearly identical across benchmarks. Verbose reasoning doesn't mean better reasoning. Why does this matter? Inference cost scales linearly with tokens. If your reasoning model generates 4x fewer tokens with the same accuracy, you cut costs by 75%. That's a massive efficiency gain. Interesting observation: Nemotron base models already had DeepSeek-R1 traces in pretraining. Training loss on DeepSeek traces started low and stayed flat. Training loss on gpt-oss traces started high and dropped gradually. They showed that the model was learning something new, which also means you can distill reasoning capabilities from frontier models into smaller systems. But the source matters. Different reasoning styles produce different efficiency profiles. (bookmark it) Paper: arxiv. org/abs/2511.19333

Post attachment
Shaltiel avatar
Shaltiel@SShmidman·25 Nov 2025

For this experiment, we used NVIDIA DGX Cloud Lepton together with NeMo framework. Lepton’s unified AI platform made it simple to scale distributed training across H200 GPUs, cutting iteration time and infrastructure overhead. Highly recommend!