←
BigScience Large Model Training129 posts
BigScience Large Model Training avatar

BigScience Large Model Training

@BigScienceLLM

Follow the training of "BLOOM 🌸", the @BigScienceW multilingual 176B parameter open-science open-access language model, a research tool for the AI community.

◷ Joined March 2022▧ 5 Photos and videos
clem 🤗 avatar
⟳ BigScience Large Model Training retweeted
clem 🤗@ClementDelangue·14 Nov 2022

The Bloom paper is out. Looks like it's doing worse than current GPT3 API in zero-shot generation tasks in English but better than other open-source LLMs & better than all in zs multi-lingual (which was the main goal). Proud of the work from the community! arxiv.org/abs/2211.05100

Post attachment
Hugging Face avatar
⟳ BigScience Large Model Training retweeted
Hugging Face@huggingface·14 Jul 2022

The Technology Behind BLOOM Training🌸 Discover how @BigscienceW used @MSFTResearch DeepSpeed + @nvidia Megatron-LM technologies to train the World's Largest Open Multilingual Language Model (BLOOM): huggingface.co/blog/bloom-me…

Saulnier Lucile avatar
⟳ BigScience Large Model Training retweeted
Saulnier Lucile@LucileSaulnier·28 Jun 2022

🌸@BigscienceW BLOOM's intermediate checkpoints have already shown some very cool capabilities! What's great about BLOOM is that you can ask it to generate the rest of a text - and this even if it is not yet fully trained yet! 👶 🧵 A thread with some examples

Post attachment

A milestone soon to be reached 🚀💫 Can't wait to see the capabilities and performance of this long-awaited checkpoint! What about you? Have you already prepared some prompts that you want to test? ✏️

BigScience Large Model Training avatar
BigScience Large Model Training@BigScienceLLM·29 Jun 2022

For 111 days, we've enjoyed world-class hardware stability and throughput thanks to the hard work of our friends at @Genci_fr, @INS2I_CNRS, Megatron & DeepSpeed. Having reached our objective earlier than expected, we'll keep training for a few more days. Stay tuned, more soon ;)