Wow, so many Twitter AI experts become material science experts over night. GPT-4 is nothing compared to human learning speed 🫠
Sadly, I'm not getting my physics PhD degree any time soon ... sorry folks, no chef's pick from me. You gotta curate your own timeline this time 😵💫
People don't buy technology for the sake of technology. They buy solutions to their problems. Similarly, people don't care how good your model is. They only care if it solves their problem.
buff.ly/454Oaud
6/ In my latest article, I delve deeper into these effects and explore the future of content in an AI-driven world. Read the full article here: buff.ly/3HTMidE
5/ New business opportunities will arise for companies that can develop and provide AI-driven services for content creation, personalization, and distribution effectively. 💼
4/ Third-order effects: With hyperpersonalized content becoming more common, platform recommendation algorithms may struggle, shifting power towards AI agents that can find and deliver valuable content to individuals. 🤖
3/ Second-order effects: The rise of personalized content intensifies competition among creators, pushing them to find their target audience and make their content stand out. This leads to the development of AI-driven content discovery tools. 🔍
2/ First-order effects: As content supply increases dramatically, the average value of content decreases. To stand out, creators must focus on personalized content, targeting specific audiences or individuals. 🎯
1/ 🚀 Generative AI is revolutionizing the way we create and consume content. As AI-driven productivity gains lead to an explosion of content creation, what does this mean for content value in today's digital landscape? Let's dive in! 👇
Oops haven't tweeted too much recently; I'm mostly watching with interest the open source LLM ecosystem experiencing early signs of a cambrian explosion. Roughly speaking the story as of now:
1. Pretraining LLM base models remains very expensive. Think: supercomputer + months.
2. But finetuning LLMs is turning out to be very cheap and effective due to recent PEFT (parameter efficient training) techniques that work surprisingly well, e.g. LoRA / LLaMA-Adapter, and other awesome work, e.g. low precision as in bitsandbytes library. Think: few GPUs + day, even for very large models.
3. Therefore, the cambrian explosion, which requires wide reach and a lot of experimentation, is quite tractable due to (2), but only conditioned on (1).
4. The de facto OG release of (1) was Facebook's sorry Meta's LLaMA release - a very well executed high quality series of models from 7B all the way to 65B, trained nice and long, correctly ignoring the "Chinchilla trap". But LLaMA weights are research-only, been locked down behind forms, but have also awkwardly leaked all over the place... it's a bit messy.
5. In absence of an available and permissive (1), (2) cannot fully proceed. So there are a number of efforts on (1), under the banner "LLaMA but actually open", with e.g. current models from @togethercompute, @MosaicML ~matching the performance of the smallest (7B) LLaMA model, and @AiEleuther , @StabilityAI nearby.
For now, things are moving along (e.g. see the 10 chat finetuned models released last ~week, and projects like llama.cpp and friends) but a bit awkwardly due to LLaMA weights being open but not really but still. And most interestingly, a lot of questions of intuition remain to be resolved, e.g. especially around how well finetuned model work in practice, even at smaller scales.
5/ Ready to explore more about the rapid progress in machine learning and its implications for businesses and other fields? Check out the full article here: embracingenigmas.substack.co…
4/ These insights can help you maintain a competitive edge in an ever-evolving market. Don't miss out on the opportunity to learn from the successes and challenges faced by the machine learning community. 🚀
3/ By understanding these factors, we can learn valuable lessons for businesses and other fields:
Foster collaboration and knowledge sharing 🤝
Shorten cycle times ⏱️
Encourage experimentation and innovation 💡
1/ Ever wondered why machine learning has progressed so quickly? 🤔 In my latest article, I dive into four key themes behind this rapid advancement and how we can learn from them to drive innovation in other fields.