←
CM1,027 posts
CM avatar

CM

@Creative_Math_

Dessa pre-training @blocks. Did pure math in a past life. cashmere-y, grad student @UofT 🍁

◷ Joined September 2020▧ 215 Photos and videos
CM avatar
CM@Creative_Math_·Apr 28

Career update: I’m joining @blocks for the Summer as a DL Research Intern! I’ll be working on some very cool and novel approaches to generative audio modelling, would be happy to get to know more people in this space!

CM avatar
CM@Creative_Math_·16h

also this, there's a call here for ppl making RL envs

Post attachment
CM avatar
CM@Creative_Math_·16h

I'm curious how (if) u can have a good knowledge benchmark with continuation models? You judge if it creates another spkr answering the question you had or sth? is it an LLM judge? also ECAPA-tdnn is a bad model, it gets only 0.8 on real continuations, the generations in the blog are better than what it suggests

Post attachmentPost attachment
CM avatar
CM@Creative_Math_·16h

I really like their blog! Some thoughts: 9M hours at 12.5Hz taking ~180k steps => assuming bf16, they trained on approx 40-50 TB VRAM (3-4 GB200 racks). This is 13 TPP in total, or 135 TPAP. Plateau at ~3.6M hours and 31B a3B, napkin math gives ~16 active experts per layer

Post attachmentPost attachment

Introducing Interact-0, the first in a new generation of always-on models. Interact-0 is a full-duplex speech model designed around human interaction. It is the only model that knows when to speak in groups, and can be active without barging in when it shouldn’t. We share our thesis, approach, and early results below. interact.inc/blog/interact-0…

2:11
CM avatar
CM@Creative_Math_·18h

I have a feeling Muon based optimizers work much better for voice modelling, I haven’t formulated thoughts on this yet but distributing your gradient across more SV subspaces should help with vocal fidelity

CM avatar
CM@Creative_Math_·Nov 16

I can’t imagine the beast anthropic can have in their RL environments now since they’re the only ones able to use Fable/Opus as a judge

CM avatar
CM@Creative_Math_·Nov 13

You see, all you have to do is [REDACTED] to fix your training instabilities and you get to enjoy this:

Post attachment
CM avatar
CM@Creative_Math_·Nov 12

Core Automation is literally assembling the avengers of optimization theory, John Langford is the core contributor for the Microsoft Dion repository (he also merged a PR of mine once ^-^)

Post attachment

John -> Core Automation coreauto.com/ After Microsoft (hunch.net/?p=13764006), I was attracted to a place which was very willing to try new things and exceptionally enabling in doing so.

naklecha avatar
⟳ CM retweeted
naklecha@naklecha·Nov 10

introducing caramel, my attempt at training a dessa (by @jack) like conversational model purely on speech. caramel sounds very human, but can't hold a coherent conversation (yet). the model + training recipe are open source. enjoy! samples of caramel talking to itself:

16:10
CM avatar
CM@Creative_Math_·Nov 9

You give a man a GPU, and you feed him for a day. You teach him how to kick his colleague’s jobs from the cluster and you feed him for a lifetime

CM avatar
CM@Creative_Math_·Nov 5

They're reinventing the concept of zakah in Canada

NDP leader Avi Lewis proposes tax on wealth over $10M This money would be used to fund 1. social housing 2. public grocery stores 3. More money for Pharmacare, Dentalcare, and Medicare

0:52
CM avatar
CM@Creative_Math_·Nov 5

As AI models deal more frequently with humans, it's increasingly important for them to have strong emotional understanding--it's a safety risk that they don't. We all imagine dystopian scenes of robots harming humans due to misunderstandings, S2S is a step towards solving that

MTS@MTSliveNov 4

Block's @RagavanThurair tested AI voice models by making sounds of pain. They thought he was stretching or waking up. He warns that this could become a serious safety problem as AI-powered robots gain more autonomy: "I tried to make sounds of pain. I had this experience recently where my toddler stepped on a sensitive part of my body, and I couldn't say words. All I could do is make sounds of pain. Imagine a Unitree robot steps on me the same way. I can’t say words to say, get off of me." "These models fail at that. It's like, oh, it sounds like you're stretching. Did you just wake up? It doesn't understand crying and things like that. There's a reason why we can hear emotion in voice." "I think it's actually a huge safety issue as well. These AI models are taking more and more control of things." "They're extremely smart, but if they don’t understand what we actually want or what we're trying to communicate, they can end up doing things we don't want." @blocks

1:37
MTS avatar
⟳ CM retweeted
MTS@MTSlive·Nov 4

Block trained its new voice AI entirely on speech, without using any text. Dessa lead @RagavanThurair says even his team didn't expect it to develop such strong language understanding: "It's trained entirely on speech, which is crazy. This company's training it just on speech, they have less data, it's a lot harder, how the hell are they gonna learn language like that? But somehow ours ended up better." "I thought we would have to have text to get to this level of conversation understanding. We didn't expect it would learn language this good." "People have done this before, like GSLM and Hertz-dev. The big gap is ours has a really strong understanding of language and you can actually have a realistic conversation with it." @blocks

1:38

Dessa is fast

0:41
Drew avatar
⟳ CM retweeted
Drew@drewocarr·Nov 4

We're gonna spend the next twenty years laughing at how embarrassing the "tech right" was these past two years.