Today is a day I’ll never forget.
I grew up in Cupertino.
My dad was a tech founder in the 80s/90s.
I was in YC S07.
LiveKit is my 5th company.
The first 4 didn’t work out.
I’ve had a lot of advantages — it still took 20 years to get here.
Founders: keep taking shots.
We learn to speak before we learn to read.
Voice is the most natural interface we have.
We just raised a $100M to make building voice AI as easy as a web app.
LiveKit has an open source version of this that you can actually USE, it was deployed earlier this month on our homepage.
Have one of the agents call you, and you'll see what I mean: livekit.com/agents
We’re bringing multimodal agents to phone calls with Call Companion: an interactive page sent by text that lets callers see and select options while talking to the agent. It gives callers a clear view of their choices, more confidence in their decisions, and an easier way to enter details that are hard to capture by voice.
See Call Companion in action. A caller compares hotels, chooses a room, and pays for her stay, all while talking to the same agent.
gemini 4 hydrogen dropped so naturally we put it into a broken voice agent to see what it would do
0 false reports and didn't fake anything
it's also faster than gemini 3.8 flash. thoughts?
full write up livek.it/BQwKvUh
it's crazy that dream-1 (@dreamlabs_ai) literally just works
i'm at home in vietnam, testing a robot in the @dimensionalos office in sf, for our hackathon with @makermodsai
it's quite surreal to see the thing works 13000km away
@livekit is sponsoring this hack, we provide a HITL reference repo for participants using livekit-portal, but it can also be used for distillation
i would love to see someone distill dream-1 to a smaller policy, then do RL in real life to hill climb the challenge (good way to win the hack)
another idea is policy escalation, you can detect failures and use dream-1 or another fine-tuned better policy to resolve failures
anyway, the possibilities are endless with portal's multi-participant capabilities.
We're hosting another Physical AI Hack this weekend, November 7–8 at @fdotinc in San Francisco.
This time, we're partnering with @dreamlabs_ai and their new world action model, Dream-1, which can do tasks zero-shot. @livekit@nvidia@huggingface@dimensionalos@enact_labs are joining us too.
@livekit is the real-time layer: connect your robot to human operators and cloud models with ultra low latency, then give it a voice agent so you can talk to it. You'll get LiveKit credits for real-time streaming and voice agents.
Other technical partners: @_lightcompany@mirdainsystems@SGhashghaei@spcxstarship
Prizes:
1st place: $2,000 + a Metal Arm + credits
2nd place: $1000 + a Maker Arm + credits
3rd place: $500 + a Reachy Mini + credits
Sign Up: luma.com/pah
There are two tracks:
Hill climb (New Track):
This is a new format where everyone works to improve the model's accuracy on the same task. We'll announce the task soon and provide the props. You'll climb the leaderboard through fine-tuning, DAgger and a System 2 agent harness around Dream-1, using LiveKit and Dimensional OS.
Creative (Zero shot):
Build the most creative project you can with Dream-1, LiveKit and Dimensional OS.
Since August, Harley, the voice agent we built on @livekit, has run 10,000+ insurance onboarding calls. Average call time is down 63%, and quotes take two or three seconds on hold, not a day or two.
My first engineering post @Harper_Insure, on how we built it: harperinsure.com/engineering…
$52M seed round. 15M+ users. But this is what we're most proud of yet.
Introducing Drama 3: the most emotive and controllable TTS model in the world
And yes, it's open-weight.
You can now control how even a single word should sound, change its tone, and shift your voice mid-sentence.
Uncle Roger was impressed too.
Try it here:
s.fish.audio/rqh0l8
Comment “DRAMA” within the next 48 hours for 1 month free.
If you're an enterprise your first month is on us:
s.fish.audio/qz9r0t
Know exactly what your agent can do, catch gaps before customers do, and test any model against your own scenarios before you switch. Your agent has a QA team now.
Try LiveKit Simulations free through October: livekit.com/products/agent-s…
GPT-Live-1 @openaidevs is now supported in LiveKit Agents. It's full-duplex, so it listens and speaks at the same time instead of taking turns, and it hands reasoning and tool calls off to a backend model so a slow lookup doesn't leave your agent sitting in dead air.
Same `AgentSession` API. New plugin: `GPTLiveModel`.
Get started: docs.livekit.io/agents/model…
LiveKit Connectors are generally available today.
They bring calls from @twilio Programmable Voice and @WhatsApp Business Calling into a LiveKit room as a regular participant, so your agent can answer there with no SIP trunk to provision and nothing new to host.
Read the full article: livekit.com/blog/introducing…
Realtime TTS-2 is GA today, now the #1 model on Artificial Analysis and fastest TTS in the world of its class.
The research preview got far more usage than expected. That experience went back into research and today's GA model represents the biggest leap in Inworld's history.