←
Kenneth Goodman406 posts
Kenneth Goodman avatar

Kenneth Goodman

@pythonprimes

Staff Engineer @ Google + Recommendation Systems Professor @ Columbia University

◷ Joined November 2017▧ 39 Photos and videos
Kenneth Goodman avatar
Kenneth Goodman@pythonprimes·Jun 30

I needed a document from Paris, it drafted a letter in French using other documents it found on my computer and brought me to payment page that prints and mails it.

using codex for personal tasks feels like using an airplane to commute to work.

Kenneth Goodman avatar
Kenneth Goodman@pythonprimes·27 Jan 2024

OpenAIs new embedding seems to really prefer earlier text. Cos(emb(t1), emb(t1+t2)) > Cos(emb(t1), emb(t2+t1)) ~97% of the time of the 1000 examples I tried. Interesting retrieval consideration

Kenneth Goodman avatar
Kenneth Goodman@pythonprimes·29 Nov 2023

With an A100 you can already get this down to ~250ms for 1 iteration

Introducing SDXL Turbo: A real-time text-to-image generation model. SDXL Turbo achieves state-of-the-art performance with a new distillation technology, enabling single-step image generation with unprecedented quality, reducing the required step count from 50 to just one. The code, research paper, and weights for non-commercial use are now available on our website. You can test SDXL Turbo on Stability AI’s image editing platform @Clipdropapp, with a beta demonstration of the real-time text-to-image generation capabilities. Learn more: stability.ai/news/stability-…

0:24
Kenneth Goodman avatar
Kenneth Goodman@pythonprimes·27 Nov 2023

I wonder if sampling of these tokens is the same as others? I’d imagine you want to be greedier with tools so it hallucinates less?

It emits special words, e.g. <|BROWSE|> etc. When the code "above" the LLM detects these words it captures the output that follows, sends it off to a tool, comes back with the result and continues the generation. How does the LLM know to emit these special words? Finetuning datasets "teach" it how and when to browse, by example. And/or the instructions for tool use can also be automatically placed in the context window (in the “system message”).

Kenneth Goodman avatar
Kenneth Goodman@pythonprimes·17 Jun 2023

What errors/mistakes/hallucinations happen 1 in 100, 1000, 10k and so on? Going to be a whole net set of outages over the next couple years

swyx@swyx17 Jun 2023

Caught gpt-4-0613 hallucinating a json value for the first time (invalid, per the schema I gave it) Ran it for ~20 times without issue. That's a ~5% error rate. Cool that we get JSON, but must be validated and rerun. We are still wrestling a beast we can't quite control.

Post attachmentPost attachment
Kenneth Goodman avatar
Kenneth Goodman@pythonprimes·14 Jun 2023

From my testing, 3.5 gets very confused if the function has a bug in it. Example of write/read to file, read file return random contents. it'll keep trying. 3.5 & 4 are convinced that it needs to re-write instead of re-read. It trusts us too much.

OpenAI@OpenAI13 Jun 2023

GPT-4 and GPT-3.5 Turbo models in the API now support calling your custom functions, allowing the model to use tools you design for it. Also — reduced pricing & new model versions (including 16k context for 3.5 Turbo): openai.com/blog/function-cal…