A tiny implementation detail in low-precision arithmetic could be biasing your AI training 😲
This interactive deep dive from Graphcore Research's @Awfidius uncovers a subtle failure mode in stochastic rounding that only appears when randomness is limited, and how it can be addressed with one simple fix 🎲
Check it out in the link below! 👇
Would you rather use 1 million × 16-bit weights, 4 million × 4-bit weights, or even 16 million × 1-bit weights?
In joint work between Aleph Alpha Research and Graphcore, we asked this question of LLMs — the answer encouraged us to embrace the wonder ✨ of 1-bit weights, which can outperform 4-bit and 16-bit weights on a fixed weight memory budget.
In our work
- ⚖️ A scaling laws evaluation prompts us to consider very low-bit formats
- 📈 Scaled-up tests show the power of memory-matched models with 1-bit weights
- ⚡ Kernel benchmarking demonstrates their feasibility for autoregressive inference
Read all about it in our blog and paper (link below! ⬇️)
Massive thanks to our collaborators at Aleph Alpha Research!
Authors: @SohirMaskey, Constantin Eichenberg, @atomicflndr and @douglasahorr
🚨 Graphcore is hiring AI Research Interns! 🚨
Join us to work at the intersection of hardware and AI and help shape the future of AI systems. Whether you're excited about efficient inference, large-scale training, or advancing frontier-model capabilities, we’ve got cutting-edge projects for you to dive into.
Interested? Apply below 👇
Have you ever worked with a remote collaborator on 3D graphics/vision and wished you could interact with the same visualizations with the full 3D context? ScenePic is great for this: zero-install, single file, animatable, with configurable layers to customize views. Check it out!
Excited to announce ScenePic: an open source toolkit in Python and C++ for easily creating rich and interactive visualizations of 3D data that run in your browser. If you regularly work with 3D data, check it out: microsoft.github.io/scenepic…
Interns play a huge part in the life of the Mixed Reality & AI Lab and make a real difference in our mission to build the future of communication & collaboration. If you're interested in pushing the boundaries of mixed reality by innovating in vision, graphics & ML, come join us!
This past year as we've all been physically separated, it has been amazing to work on bringing Microsoft Mesh to life, and imagine ways to bring people together even when we're apart youtube.com/watch?v=Jd2GK0qD…
Another great event is coming up, and this time Datagen and Machine & Deep learning Israel are partnering to bring you an event all about Synthetic data!
Check out this impressive list of speakers :)
Link to register: https://t.co/9xtrKO4hWQ
Internships in our team are a brilliantly creative and educational experience, particularly for the project mentors! I learnt so much from this year's interns: can't wait to see what we build with the students who join us next year.
Are you a PhD student interested in pushing the boundaries of Augmented Reality? Neural rendering? Holoportation? Come join us as an intern at the Mixed Reality & AI Lab in Cambridge, UK!
Find out more and apply here:
careers.microsoft.com/us/en/…@HoloLens@MSFTResearch@Microsoft
Here are the latest Pillar 2 capacity and tests. The capacity figures haven't been published since 10 August (two days after the Head of Testing and NHS Test and Trace made her statement). My question: what is the current capacity in Pillar 2, and how is this figure calculated?
When Jingjing and @_tim_hutton_ proposed this idea for optimizing the HoloLens 2 hand tracker, I was convinced it wouldn't work. They proved me wrong and gave us 4x faster surface evaluation. Building a product that our customers love required a blend of engineering and research.
The Phong Surface: Efficient 3D Model Fitting using Lifted Optimization, one of the key innovations that make the fully articulated hand tracking, running in real time in 4 gigaflops of compute on the HoloLens 2. @HoloLensyoutube.com/watch?v=YdDLW4UD…
ECCV20 oral live at 16:50 today
A little song parody project that Mrs C, @simoncragg and I have been working on. Enjoy! Stay in your house or flat (Men in Black Parody) youtu.be/MhjYeYBCbFA via @YouTube
Fantastic to see the work the HoloLens team is doing to support members of the VentilatorChallengeUK consortium to set up new production lines for ventilators news.microsoft.com/en-gb/202…
This is the most convincing description I've read of a near-term exit strategy. Get outbreak under control ("the hammer"), maintain R~1 and buy time ("the dance"). Coronavirus: The Hammer and the Dance by @tomaspueyolink.medium.com/gIH5T8WBZ4