
randomly thought about perl one-liner challenges this morning. another era.

a tiny schemer. engineering @trydaily and @pipecat_ai core maintainer.

randomly thought about perl one-liner challenges this morning. another era.

Pipecat 1.16 adds PerceptionReader, something I'm really excited about. How someone speaks can matter just as much as what they say, and voice agents have mostly been missing that. We’ve been testing it with Google Decision 1, released yesterday. Stream it audio and you get a pretty accurate read on the prosody, tone and sarcasm of what the user said. Try talking to it at this demo here: pipecat-google-decision-demo… We've exploring many different ways of measuring expressivity... more to share on that soon! ➡️ Pipecat Docs: docs.pipecat.ai/api-referenc… ➡️ Google Decision 1: blog.google/innovation-and-a…

This was long overdue! For all the Schemers and macOS users out there: I've updated the Guile Homebrew Tap with the latest versions of most of its libraries. Happy hacking! github.com/aconchillo/homebr…

Building better voice AI should also mean giving developers more choices. Today at #deepgramspeak, @trydaily, the company behind the open source @pipecat_AI framework, is joining Deepgram. We're thrilled to welcome @kwindla, our new Head of Open Source, and his team to Deepgram. We’ve also co-developed the Real-Time Context Standard with @pipecat_AI, developed to help voice AI systems carry the context that makes conversations work. More on the vision we’re working toward together: deepgram.com/blog/daily-pipe…

I can't express how excited I am about this and the future of Pipecat.
kwindla @kwindla Oct 29 Article Deepgram is acquiring Daily and establishing an independent Pipecat Foundation I'm excited to share that the Daily team is joining Deepgram. We will continue our work on the Pipecat open source framework and on our global network infrastructure that powers realtime audio, video,




is it ok to say that working on @pipecat_ai is still so much freaking fun!

New Pipecat UI component: duplex timeline. I've been using this in dev for a while to visualize turns, latency, interruptions and back channeling. ui.pipecat.ai/docs/component… Our shadcn registry is "pipecat", if you're a CLI person. npx shadcn@latest registry add @/pipecat

A couple of weeks ago, we released a demo with Pipecat and Unreal Engine: youtu.be/xfyRDiv5eWQ Some people asked, "What about Godot?" Well, it's Godot time! Happy haunting! 👻 github.com/pipecat-ai/pipeca… @pipecat_ai @godotengine @UnrealEngine

This is awesome. Multi-character interactions in Unreal Engine, completely LLM-driven and unscripted. You can spin your own version of this up pretty easily. This is another Pipecat + Jev experiment from @JonPTaylor. Watch Jon walk around the game world, talk to the characters, and ask them to do things. They interact with Jon, with each other, and with the world. Tech stack: - One Pipecat agent process syncing state with Unreal via the Pipecat RTVI protocol. - Three non-player characters. Each character is a Pipecat worker with its own LLM prompt and tools. Each character also has its own TTS service so its voice is unique. - When Jon (the human) talks, Jev assesses Jon's intent and which character(s) he is talking to. - Unreal 5.8 with Spatial Audio. (Watch the video with headphones on!) - Pipecat C++ client embedded in Unreal Engine.

Probably the best battleship game ever made. 🔥 🚢
NVIDIA released a fast speaker-labeling model today: Nemotron 3 Diarization. Like all of the Nemotron family from @NVIDIA_AI, this is an open weights model, so you can grab the weights from @huggingface and use the model in the cloud or on your local device. Nemotron 3 Diarization is a useful building block for voice agents and realtime AI applications that need to handle speech from multiple people at the same time. It handles overlapping speech well, can identify up to eight speakers, and pairs with any transcription model you're using in your multi-modal pipelines. @jonptaylor built a demo (code below) and recorded a technical deep-dive showing how to use this diarization model together with a speech-to-text model. Jon's code also uses Jev for super-accurate confirmation of critical voice input in a noisy/multi-speaker environment.

Audition mode is coming along in Jamcat. The generated midi guide is used in audio-to-audio to generate stems that match your voice prompt. Each instrument type needs tuning and training, but it's not far off!

Here's a complete voice agent tutorial focusing on a super low latency configuration: Pipecat PhoneLLM Alpha 1 on @modal, Deepgram transcription, Cartesia voice. You can grab the code and jus run /setup in Claude or Codex. But Merve's video also dives into the details of configuring model endpoints on Modal, the Pipecat bot file, and the front-end UI code in the example project.

Introducing PhoneLLM, an open model for voice agents. GPT 5.6 Terra performance on typical voice agent tasks at 1/3 the latency and 1/18 the cost. For voice agents, we need models that are both very low latency and very good at tool calling and instruction following. There's a trade-off here, and we often have to compromise on either latency or capability when building voice agents. With PhoneLLM (and the training and data stack that made this model possible) we're fixing this problem. For the last couple of years, most of the effort in frontier model development has gone towards leveraging test-time compute. Which is awesome! Models of all shapes and sizes are available that perform really, really well ... if you have "thinking" turned on for your model. But if you need your agent to respond at voice conversation speed, you can't use thinking models. PhoneLLM is a full-weights fine-tune of NVIDIA Nemotron Nano 30B. We trained on a wide range of real-world telephone and customer support use cases. The training focused on taking the excellent Nano 30B base capabilities and teaching the model to do typical voice agent tasks with thinking disabled. The results are really good: accurate tool calling and concise, on-topic responses in long conversations. And fast: TTFAT measured server-side is <100ms if you run PhoneLLM on a lightly loaded B200. :-) But seriously, when we characterize model latency, we do it with full, end-to-end, batched request simulations using real Pipecat voice agent pipelines. You can serve more than 80 concurrent agents on a single B200 with P95 end-to-end TTFAT <600ms. Including network overhead. That's an LLM cost-per-minute around $0.0025. (1/4 of a cent.) At a latency lower than any third-party API offers today. More details about this model, including weights on @huggingface, how to spin it up with one click on Modal, and a starter project repo you can clone, are in the thread ...

Congrats to @DeepgramAI on the GA launch of Flux TTS, now live in Pipecat 🎉 @JonPTaylor looks at how Flux TTS delivers a more consistent voice experience. Flux reads the whole conversation, not just the next line — adaptive tone, consistent pronunciation, clean interruption handling natively (no SSML markup, no style tags!) all at sub-200ms latency. ➡️ docs.pipecat.ai/api-referenc…

World Cup Voice Commentary right in your MenuBar! @pipecat_ai powered voice bot, not only reports on the scores but can answer any questions I have about the games! Using only local models on my M4 Max. Under the hood: + Kokoro TTS for commentary + Gemma 4e4 (with and without thinking) + Nemotron 3.5 for ASR Built with Pipecat subagents: when the menubar starts, it spins up a main loop of [STT] | [LLM] <-> [subagent when a game starts] | [TTS] There is one subagent per game. Schedules are pulled regularly, so I can ask stuff like “Who will the US play in the round of 32?” and have it reason over fixtures, for example. A collector ingests live commentary (per match-id, home/away, and metadata) into a local DB and exposes it via a CLI so pipecat's subagents can query it. Each game subagent gets an initial prompt about “its” match and tools including an /analyze-game-skill. The skill was built and tested with the early R1 games, especially the chaos of Canada-Qatar game (with injuries, goals, and two yellow cards becoming a red!) The result: when something happens in the game, the TTS chimes in, excited or dejected, about missed chances, corners, cards, etc. Subagents maintain their own context, so with push-to-talk, which invokes the STT, I can ask about what is happening in a current, past, or future games, fixtures, etc. The main LLM routes to the correct subagent, or can directly check via CLI, or spins up a new subagent to go hunt down answers (e.g., scores about R1 games, schedule changes, etc.) Today’s rain delays accidentally stress-tested having multiple concurrent games in Round 3, and the subagent orchestration held up nicely. Clips + screenshot below 👇 Halftime announcement, second goal, and the third goal. These were voice memo recordings, aargh