
essays

context maximiser @ArbResearch, prez @ paradigm3.org

essays

BREAKING: Eighteen months after writing about my search for Nathan Fielder’s long-lost feature documentary LOVE AND CAMERAS IN AMERICA (2008), the company that holds the rights just e-mailed to say they've digitized the film and posted it online (!!!) theglobeandmail.com/culture/…

AI is indeed a staggeringly impressive research tool, a Google on steroids (just as Google was a card catalogue on steroids), *so long as you check the results*. And not just check them to see if they’re correct: the sources the AI draws from almost always offer a far richer answer to a query. This isn’t that different to best practice with Wikipedia. So long as one stops here, and doesn’t use the AI to write for you, I don’t feel there is any cognitive offloading or immoral behaviour. But there is a grey area with respect to *discussing ideas* with an AI. Here, depending on the research topic and the task, cognitive offloading and moral quandaries can begin to occur, but in other cases it’s absolutely fine. It is in this grey area where society needs to pay closer attention to our moral intuition (I.e. everyone already knows when they’re cheating, misrepresenting, taking shortcuts, or lying). Such moral intuition also simultaneously happens to offer a good regime for maintaining cognitive fitness.
I never write with AI, and advise my students not to. Before long I think there will be advantages to being a clearly AI-less writer. But I want to be explicit about one change of mind. As a research tool, I have had to recognize its power. Ironically, this happened because I am trying to learn about AI as fast as possible. A trusted correspondent said in response to a semi-technical question, "That's the sort of thing I just ask Claude, and then check the answer." The results (as countless others have found) were impressive. I have come round to this kind of AI use. I not any less concerned, though, about its use in composing text.




Examples of things we're doing: * Publicly commenting on every serious pacing proposal substack.com/@pacingtech * Adapting the 'Intelligence Rising' wargame into an AI environment intelligencerising.org/ * Plotting the frontier of verification strength vs noninvasiveness

When you don't know what to do, run a fellowship If you liked our pacing policy agenda (pacing.tech), come do research with us. Paid, remote, not boring.

More and more people are calling on U.S. frontier AI companies to pause, but when should they actually do so? I try to answer this question in a new blog post, summarized neatly by the two attached images TL;DR: A lengthy pause now would be premature, but shorter pauses tied to addressing specific safety problems are likely good link below

i kinda love how Tasmania has a “Godzilla” like relationship with this guy. “yeah it comes ashore and destroys as much as it can every once in a while but you know, it’s like a spectacle and it leaves eventually”
Tasmania’s notorious ‘Neil The Seal’ is back on land & wreaking havoc 🦭 [via/ X @jonzzy ]

I recently became aware of how confused even many academics are about Pangram. Please, please, look at the audits that have been done before you deliberately sow doubt. (link in thread)

This is just... not true? Edison famously published his predictions for the phonograph in an 1878 essay and this was his list below. It was funky because it assumed letters would be the primary focus, but it wasn't all that wrong. And in a way it was quite visionary. Either way, sermons are no part of it. (Edison himself was a deist, I believe?) Edison top predictions for the use of the phonograph, in his own words: Letter writing and all kinds of dictation without the aid of a stenographer. Phonographic books, which will speak to blind people without effort on their part. The teaching of elocution. Reproduction of music. The "Family Record"--a registry of sayings, reminiscences, etc., by members of a family in their own voices, and of the last words of dying persons. Music-boxes and toys. Clocks that should announce in articulate speech the time for going home, going to meals, etc. The preservation of languages by exact reproduction of the manner of pronouncing. Educational purposes; such as preserving the explanations made by a teacher, so that the pupil can refer to them at any moment, and spelling or other lessons placed upon the phonograph for convenience in committing to memory. Connection with the telephone, so as to make that instrument an auxiliary in the transmission of permanent and invaluable records, instead of being the recipient of momentary and fleeting communication.
Marc Andreessen explained why the people who invented AI are the worst people to ask about its future, using a mistake Thomas Edison made in 1877... Geoffrey Hinton won a Nobel Prize in 2024 for the work behind modern AI. When people ask Hinton where AI is going, he believes "rampant unemployment" and UBI for everybody. "People think, because he's one of the inventors of AI, that he must be the guy who knows." Andreessen's answer is Thomas Edison. Edison invented the phonograph in 1877. To him, Andreessen says, the use was obvious. Every family would buy a record player and a library of discs with sermons from the great preachers of the time. Then the machine went on sale. "Immediately, of course, it's music. It's ragtime, and swing, then it's jazz. And Edison's just completely horrified." "If Edison didn't know what the phonograph was going to get used for, the idea that Joe AI entrepreneur is going to be able to forecast the economic implications... No. That's not going to happen." Inventors, he says, are "too buried in the specifics of the here and now." He includes himself. "I don't know if there's anybody that can predict big cultural, or economic or social trends, but it's certainly not somebody who's been in the lab for 20 years, including myself."

Our new issue features a drive-by statement of my view of AI progress, which I hope to write about more soon

"it sounds to me like these auditors can freely talk to anyone at the company 'off the record'" -> This is not the case, we can only talk with company-approved staff. Additionally, we cannot quote or summarize info even from approved staff. Sorry if we didn't communicate this well.
i agree that this does not seem employee like and therefore does not live up to the original premise. it’s an odd compromise that’s 80% of the way there in terms of employee like access but baroque enough that it gets 20% of the public credit. what anthropic outlined with their METR deal seems more straightforwardly 'employee like' (at least as far as publicly known) that being said i know how hard many people worked to get even to this point, and imo it will accomplish all of the strategic objectives for "pacing the frontier" 1.0 by at the least verifying where the frontier is. the assessment contracts will grow quickly past "first phase" it sounds to me like these auditors can freely talk to anyone at the company 'off the record'. for stuff that informs the public reports it then needs to go through official channels. this is how i understand embedded auditing works in other settings separately, I actually do think openai is quite transparent with misalignment reporting / RSI progress / and so on, and there is a broad oversight mechanism that comes from the nonprofit board Paul Christiano et al that may not be obvious on the outside


because you have not heeded my warnings and learned Spanish yet, The Point has republished in English my essay on Pangram, AI writing, the third world novel, and why so many humanists have chosen ignorance instead of grappling with what AI means for literature thepointmag.com/examined-lif…