what there is no overwhelming agreement on is what will happen the day after. how do we build a livable future here, for both israelis and palestinians. israel leadership is not thinking of this now, future looks grim. THIS is where protest effort and outrage should be invested.
אני לא מסכים. AI זה לא ML. כלומר כן, הדרך שבה AI מודרני נוצר כוללת בתוכה רכיב משמעותי של ML. אבל אחרי שה-AI (ה-LLM) נוצר וקיים הוא כבר לא ML, הוא משהו אחר וחדש. התמחות נפרדת.
private evals don't exist. once you evaluate an entity (say, OpenAI runs GPT6 on your eval), the evaluation set is no longer hidden from them (they could use this to train GPT7).
re "openness in AI" discussion: there is a lot of talk about the importance of open-weight models. I want to highlight another topic: openness of APIs.
LLM-access APIs were traditionally mapping in a relatively straightforward way to what is actually sent to the LLM: a sequence of messages, each with a different role, and each call specified the exact sequence. this gives both some level of transparency into the underlying model, and also a level of control of our interaction with the LLM.
both Anthropic and OpenAI are attempting to transition into alternative APIs, ones where the session state is managed on the server. currently they support both versions, but i predict a not-so-distance future where they will be deprecating the old APIs and leaving us only with the new ones.
this will add a layer of obfuscation between us (the API users) and "the AI" (no longer just an LLM, we will now communicate with an opaque system). This will be very bad for science, engineering and progress around AI: we (the public, the research community, the builders) will have much less visibility into what is actually being run, and also much less control on where we can intervene. this means research on agentic-harnesses and context management, for example, will be practically impossible.
this is a bad outcome. we should be able to resist it, or at least to loudly signal against it. i think it is as important as supporting open-weight models, if not more.
how do we organize around this cause?
is your feed also buzzing with this "pybot" thingy? the "600k of agent-swarm-written chatbot which is just rules, no model, no weights, no training"?
i just looked at the repo. it is junk. similar to what you'd expect from a mediocre rule-based chatbot. but somehow its 600kloc
in this post the Claude-Code dev team boast that 90% of claude-code code is written and verified by agents, without a human review. and then they provide some generic tips about what are good tests etc.
but my take is that the real reason they can pull it off is not their amazing auto-testing skills, but the fact that claude-code is essentially a very simple piece of software, they don't have almost any efficiency concerns, no competitors, very little potential of actually harmful bugs or data corruption due to claude code itself, and that many bugs just got swallowed and compensated for by the underlying LLM being so freaking good.
Claude writes better code when it can verify its work. Checks that used to be expensive, like tests, benchmarks, and end-to-end runs, are now cheap.
Here’s how the Claude Code team thinks about verification:
claude.dev/blog/how-the-clau…
i still stand by this btw.. it is silly and creepy to reject because a text was LLM-generated. what i dont want is to review generic pompous but empty texts, or to review papers without a clear narrative, or papers that are just junk, or papers with the LLM style that make me want to puke. we can definitely reject on that, ideally desk reject with a very permissive rejection policy. but the problem in these is *not* the fact that LLM participated, but that not enough human participated, and/or that the result is junk.
i think jensen huang was a strong ceo of an important company and that there is a high likelihood that nvidia will do much worse without him
but come on if he really cared about "openness" and not just as means of selling more nvidia products, how about opening the CUDA APIs
it cannot be because of his sex life: this explanation is just not consistent with any of our prior observations of how the world works. so what *is* it about? what does the board know?
"""
subj: Your paper: [title]. a request
body: I read your paper [title] with great interest. It explored the topic of [topic] from an interesting angle. I am also applying for a US visa and I need a support letter. Can you write one for me?
"""