Founder, CEO at @PhysicalAI. Tech leader and executive, interaction designer, scientist. Google, Disney, Sony before. 2019 National Design Award. TED speaker.
Archetype AI has partnered with NSW to bring Newton to Japan, and this collaboration means a lot to me. We're starting with manufacturing and energy. Newton runs on edge devices and on premises, so customers keep control of their data.
bit.ly/3VqQtb3
Some of the most exciting work on Newton right now is happening in healthcare, with sens8ai and Dr. Archana Dubey. Their new health and wearable sensing platform, Darwin, is built on Newton, so the team can put its clinical expertise where it counts most.
Tomorrow I join Hiroaki Kitano of Sony on a @wef panel about world models. We worked together at Sony CSL in Tokyo when I was Research Scientist in @rkmt group years ago and I’m excited about this reunion.
AI has read everything about the physical world and experienced none of it. World models change that.
Nov 12, 6am PT. Learn more: initiatives.weforum.org/stra…
AI is getting very good at noticing what happens in the physical world and still bad at explaining why. Our research with @encord_team found that gap in medical, HVAC, and climate data.
Thanks @connectedworld for writing about why it matters.
connectedworld.com/ais-next-…
Read our latest research blog: bit.ly/3UFCbD7
This is a little bonsai that grows differently depending on the room it’s in. In a hectic meeting room, it grows lots of branches of different sizes. In a quiet studio at home, it grows fewer branches, some bigger than others.
We’ve been running a few experiments to see how physical AI could help everyday objects adapt to their surroundings. This is the first one, built by @lukas_moro.
A closer look at the functionality of the 'little tree'. 🌳
An @adafruit Feather nRF52840 Sense board streams light, sound and vibration to Newton Omega, our time-series model, running on a @Raspberry_Pi. When the blinds go up, it sees a new state and grows a branch. In the end you can see the embedding map with Newton's outputs. After a state is not detected for a while the branch yellows and drops off after about ten minutes.
Moving on I will add many different states the tree can go through and build a game director where Newton facilitates the long term growth of the tree and short term reaction like wind, blossoming and more.
We showed Newton, our physical world model, a motor it had never seen. With no labels, it separated off, on, and an anomaly from acceleration and current data that we provided.
At the last week's Strange Summit hosted by @taratan, our co-founder and Chief Scientist @_jaimelien_ explained how it works: Newton is a physical world model pre-trained on sensor data. It can learn how new machines it has never encountered behave instead of being rebuilt for every asset. One model across the fleet, deployed on-prem, at the edge, or in your own cloud.
Learn more about Newton capabilities: archetypeai.io/blog/adaptive…
Introducing Encord-REACH: an end-to-end pipeline for real-world egocentric hand tracking.
State-of-the-art 3D hand-pose accuracy, near-perfect wearer/bystander tracking and 30–70% lower error on industrial gloves.
See it in action ↓
This is what I’ve been saying lately: eval is the bottleneck especially in Physical AI, not models.
Want to automate manufacturing? Prove your model works. The data to do that is locked inside customer systems, and they ain’t sharing it. Neither synthetic data nor contrived tasks like robotic clothes folding can replace validation on real industrial problems.
You earn access by doing the work: deploying on customer infrastructure and solving their real problems alongside them. That knowledge is your moat.
Databricks Chief AI Scientist Jonathan Frankle (@jefrankle) says the bottleneck to enterprise AI adoption is evals, not models.
"The models are good enough at a lot of really complex enterprise tasks and knowledge tasks"
"the real challenge is describing what you want AI to do and knowing whether AI did it"
I have been spending this week at the NTT Data Open Innovation Week in Malaga, Spain. Together with our lead creative engineer, we have been working with NTT customers and the NTT team in rapid development sessions to solve hard customer problems and move towards production. It is super intense, but it is great to go deep on the customer use cases and figure out how Newton physical world models can solve hard challenges for customers.
Newton’s embeddings turn a window of raw sensor readings into a single point, placed so that moments that look alike in the physical world land near each other.
This public dataset, PAMAP2, shows data recorded when a group of nine people wore three motion sensors, on the wrist, the chest and the ankle, while going through twelve everyday activities.
This visualization tool helps our team see the data in more detail and isolate different states: Rest, walk, run, cycle, stairs and housework each settle into their own region.
Seeing why a model groups two moments together is the first step toward understanding what it tells you about the physical world.
Built with Newton on the PAMAP2 physical activity dataset (Reiss and Stricker, 2012).
When people hear Physical AI, they picture a humanoid robot. Our CTO, Nick Gillian, makes the case that most of the real opportunity lies in existing equipment rather than in deploying a humanoid workforce.
Every factory is streaming radar, video, vibration, and current. Newton is one model that reads all of it, catches what’s off in real time, and adapts to machines it’s never seen, right at the edge.
Full conversation with @AI_in_the_AM ↓
“The washing machine has changed the world more than the internet has.”
— Ha-Joon Chang, 23 Things They Don’t Tell You About Capitalism
Imagine what Physical AI could do when it automates everything.
Physical AI > Digital AI.
Even more unexpectedly, Newton RF also performed strongly on non-RF tasks like vibration, audio, EEG, and seismic data. That means we should be able to expand Newton into the one model that understands any physical phenomenon, reads any sensor, and works for any use case. One world model that encodes the entire physical world.
Full research: archetypeai.io/blog/newton-r…
Newton now understands radio frequency (RF) data!
We just released results comparing a new Newton RF model against specialized RF models. Newton RF was trained on a large volume of RF data that looks nothing like other physical measurements, and we improved the model architecture to capture the spatial signal characteristics inherent to radio frequency.
The results: Newton RF beat all five specialized RF models in 18 of 20 radio tasks, including localization, channel modeling, and blind demodulation.
This is remarkable. In RF, models are highly specialized, each carefully designed for a single task: one model to decode, one to localize, one to identify the transmitter, and so on. Jonah Casebeer and @_jaimelien_ have proved that our Newton world model can rule them all!
Say “AI in the physical world,” and everyone pictures a robot. The robot is the least interesting part of the story.
The opportunity goes far beyond robots: models need to learn from real sensor data, adapt as physical systems change, and run on the edge and offline.
Thanks to @TechCrunch for having me and giving me the stage to discuss what it will take to bring AI into the physical world with @ethanjb and @IanRountree.