Been thinking a lot about UIs lately while doing customer research for @join_atlas_ai and I've concluded that UIs almost always get worse over time. Companies may hire better designers or PMs to avoid this, but their UIs will inevitably become harder to use anyway. Here's why.🧵
I'm back-testing Opus 5.5 medium on old sessions to see how many fewer prompts they'd need. Claude takes a first pass at which original prompts the replay would still need, and I'm correcting its calls in a session explorer I built.
Below are some initial data points.
All models, no matter how smart, will eventually build systems that they can no longer understand or maintain, if you let them. Fable 5 finally outbuilt itself, and flailed on me for a week. Fable 5.1 looks like it will fix it. For now. But you have to keep an iron grip on system size, or it'll run away from you.
Companies that merely ship the code Claude writes promoted by its marketing copy / sales pitches and guided by its product strategy will be a commodity.
Anyone can ask Claude.
Valuable companies in the future must be built by people who can find where Claude is wrong.
The new METR time horizon graph is pretty bad imo. It's a great benchmark, but the time horizon estimation isn't reasonable rn. I think something like this would be more justified:
Today's your lucky day, folks. @join_atlas_ai is winding down, which means @JoshShroyer -- an awesome engineer, person, and friend -- is on the market.
I met Josh 8 years ago while working at another startup. It was immediately clear to me that he was a brilliant engineer. Software was a craft that he cared deeply about and he studied with a fervor that I've haven't seen in anyone else in my decade long career.
Josh's humility was equally apparent when we first met. Although he knew more than his peers, he was cautious in his thinking and patient with how he communicated.
As I got to know Josh better, I learned that in addition to being humble and brilliant, he is also a ridiculously kind person. He really cares about people. He wants to do right by the users and customers he builds for.
After we parted ways, I knew I wanted to work with Josh again, so I sought him out after raising capital for my new venture. He has continued to impress me with his ability to quickly pick up new technical skills. He went from iOS dev to fully productive Fullstack web and AI dev in less than a month. He pushes himself hard, and it shows.
Although he works extremely hard, he has remained kind, patient, and calm through all the ups, downs, and twists of startup life. I'm grateful that I had the opportunity to work with him twice, and I'm especially grateful that someone so talented and kind put up with my first crack at being a CEO of a startup.
This isn't my last attempt at being a CEO, so if you hire Josh, treat him like royalty because I'm definitely going to try convince him to work with me again!
Trying to get a sense of developer attitudes about data privacy. Please pick the option that best fills in the blanks in the sentence below:
I am building a (B2B | B2C) product and am (happy | unhappy) with our privacy practices.
e.g., pick choice 1 if you're working on a B2B product and are unhappy with your product's data privacy practices.
Trying to get a sense of developer attitudes about data privacy. Please pick the option that best fills in the blanks in the sentence below:
I am building a (B2B | B2C) product and am (happy | unhappy) with our privacy practices.
e.g., pick choice 1 if you're working on a B2B product and are unhappy with your product's data privacy practices.
We've reached the point in platform maturity where businesses need to concern themselves with building for a niche, or delivering features @Apple won't go after, else risk feature absorption
@1Password is a good example, growing out of Apple's ecosystem.
techcrunch.com/2024/06/18/io…
PSA: gpt4o's behavior is chaotic and unpredictable when temp is set just above the default (of 1.0). [0]
Past temp of 1.1 we're seeing nonsense responses in multiple languages and character sets
[0] platform.openai.com/docs/api…
There's consensus emerging among VCs that #LLMs will make software companies look more like services businesses because software will own a task end-to-end instead of merely enabling a human to work on a task faster. I've seen this from partners at @NFX , @FoundationCap, and @OpenViewVenture. The shorthand they like for this "LLMs enable service as a software."
I'm betting the company (@join_atlas_ai) that they're wrong. I'm trying to work with GTM/PLG teams instead of replacing them, not because I'm particularly precious about displacing jobs but because LLMs just aren't good enough to do their jobs and that won't change anytime soon.
LLM optimists point to LLM's effectiveness in domains like coding, prospecting, and SEO content gen, but the wins here are limited and will be short-lived.
Coding is unlike any other domain in that the content (code) generated by LLMs can often be unambiguously verified as correct. The code either runs or it doesn't. It's a mistake to generalize from the coding domain.
The effectiveness of LLM-powered SEO content gen and prospecting, moreover, is primarily a function of the competitiveness of the space in which these solutions operate. Once the competition shows up, no one is going to care about 80th percentile performance on cold outreach or SEO content. Humans will be back in the game. E.g., Dave Rigotti banned GPT for content gen at @Inflection_Io this week. I already use "better than ChatGPT" as a filter for content I want to write.
LLM optimists often think that there's a clear path to making these models much better. There isn't. Pay less attention to people who have a financial interest in having you believe otherwise. Don't take my word for it either. I'm just incentivized in the opposite direction. @GaryMarcus is too, but he's worth paying attention to.
Anyone else notice that #LLaMA3 performs differently on @GroqInc's API than it does locally?
I’m running llama3-8b-8192 (instruct) via @ollama on an M1 mac and seeing consistently better accuracy than the same model on groq for the same tasks (mostly 0-shot text classification)
@stripe cc @jeff_weinstein low hanging UI improvement that would help new dashboard users: semantic search over your UI
would match queries like "add user to account" to the page the user intends to go to. You shouldn't need to know the magic word "member" to get there