←
James Johnston2,043 posts
James Johnston avatar

James Johnston

@P2PJim

CEO @Picloenergy, writing about energy (grid flexibility, communities, regulation etc) and post-agi society (work, welfare, purpose etc).

◷ Joined October 2011▧ 47 Photos and videos
Casey Handmer avatar
⟳ James Johnston retweeted
Casey Handmer@CJHandmer·Nov 17

In my ongoing quest to find the limits of Opus 5.5, I directed it to audit my old scientific field, theoretical astrophysics. We indexed about 150,000 papers, audited 50,000 for issues, and did progressively more thorough checks on thousands more. We found nearly 1000 major errors that got cited many times, most are still uncorrected. This survey enabled me to estimate the efficacy of peer review for correcting errors (not very good) and to identify dozens of open questions that anyone with spare Claude Max tokens could answer today (.md context files included). Bottom line: AI can speed up quantitative science by >1000x with no further model improvement required. But I'm seeing no evidence of this being pushed within universities, only academics complaining about labs dropping math proofs. Among my fellow reformed academic high agency founders, however, several are speed running entire disciplines. This survey has implications for how science will be run in the very near future. Hint: Academic publishing is dead. It was already a very stupid way to record and promulgate knowledge, and had been so for decades, even before all that publicly funded knowledge was locked behind paywall cartels. Now AIs can generate thousands of novel, correct papers per day, it is no longer meaningful for humans to consider a research paper a "unit of science" or to even bother much with the 99.9% of science which, up to this point, was necessary drudgery. Time to discover something like General Relativity or Evolution every week. LFG!! Enjoy: science.caseyhandmer.com/phy…

Post attachment
Jensen Huang avatar
⟳ James Johnston retweeted
Jensen Huang@JensenHuang·Nov 17

Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen

Alex MacCaw avatar
⟳ James Johnston retweeted
Alex MacCaw@maccaw·Nov 12

I feel like we haven't properly internalized this graph.

Post attachment
DHH avatar
⟳ James Johnston retweeted
DHH@dhh·Nov 13

Local AI. Local compute. Open weights. Open source. Global collaboration.

Chase Dowling avatar
⟳ James Johnston retweeted
Chase Dowling@cpatdowling·Nov 10

450MW of distributed batteries represents a meaningful portion of CoServ's peak load management. Nothing like this has ever been built at this scale, integrated into an existing power system. It's an incredible opportunity to work with CoServ building the grid of the future on the foundations of a century's worth of power systems engineering.

Eight months in, we’re expanding our partnership with @CoServ to 450 MW of distributed capacity across the co-op’s service territory in Dallas–Fort Worth. Base has already installed 75 MW at CoServ members’ homes, adding roughly 10 MW each month. The expansion builds on that momentum, giving CoServ a larger resource to dispatch during peak demand while more members receive affordable whole-home backup power. Read the full announcement: businesswire.com/news/home/2…

Joan Larroumec avatar
⟳ James Johnston retweeted
Joan Larroumec@larroumecj·Oct 27

Voici ce qu’il va se passer. D’ici 5 ans, 2 ans d’après les plus optimistes, les IA sauront faire à peu près tout ce que 99 % des humains savent faire aujourd'hui. D’ici 15 ans, peut-être 10 d’après les plus optimistes, les robots sauront faire à peu près tout ce que 99 % des humains savent faire aujourd'hui. Mais cela ne veut pas dire que tous les humains seront remplacés dans leurs emplois instantanément. Pour trois raisons : 1. La société sera longue à s’adapter. Tous les process d’entreprises, les masses salariales, les procédures RH, les organisations d’usines, les méthodes de travail, etc. devront se réinventer. Cela prendra du temps, des consultants, créera beaucoup d’hétérogénéités entre entreprises. Une grande part de l'organisation sociale actuelle repose sur des conventions, pas des nécessités économiques. Pléthore de gens doivent leur emploi à l'inertie des institutions, au fait que les chefs veulent de grosses équipes, que c'est souvent plus facile d'embaucher que de licencier. Beaucoup d'entreprises ne s'adapteront jamais, et continueront tant que leur rente de situation leur permettra, avant de disparaître et se faire remplacer. Ce processus pourra durer longtemps. 2. La société ne voudra pas s’adapter. Non seulement les lois limiteront beaucoup d’usages (en médecine, dans les professions réglementées, dans l’éducation, le recrutement, la justice, les infrastructures critiques, que l’AI Act européen classe déjà « à haut risque »), mais aussi de grands mouvements luddites réclameront plus de lois. On a l’exemple de ces dernières semaines des mathématiciens : vingt-cinq médaillés Fields ont dénoncé dans Le Monde, le 11 septembre, un « grave désalignement » de l’IA en mathématiques, et plus d’un millier de chercheurs réunis dans l’Association for Human Mathematics se sont engagés à ne pas travailler pour les entreprises d’IA ni à publier de texte généré par une machine, une partie d’entre eux renonçant à tout usage de l’IA dans leur recherche. On sait bien que ça n’empêchera pas les Américains et les Chinois de les rendre obsolètes en quelques années, mais ces résistances locales ralentiront la transition. Il y aura des centaines de pétitions anti-centres de calcul, et des milliers de zadistes qui mettront le feu à des grands projets d’infrastructure IA. Bientôt notre quotidien sera fait de tribunes et d'actions anti-IA. 3. Les contraintes physiques ne se relâcheront pas tout de suite. Il va falloir beaucoup, beaucoup d’électricité, de centres de calcul, mais aussi d’usines à servomoteurs, actuateurs, d’acier de précision. Et construire, ça prend du temps, ça demande des permis, des négociations. Construire des centrales nucléaires en masse, ça prendra 20 ans, avoir un écosystème qui produit à l'échelle, sans doute autant. Mais il faut se rendre compte d'une chose : me plus probable est que le monde soit déjà engagé sur un chemin irréversible. Nous sommes déjà tous morts (en tant qu’organisations sociales) mais beaucoup ne le savent pas encore. Tout comme une fois que la révolution industrielle avait commencé à Manchester à la fin du XVIIIe siècle, les sociétés traditionnelles partout sur terre étaient déjà mortes sans le savoir. La Chine des Qing, la plus grande puissance mondiale de l’époque, n’est morte qu’en 1912. Mais elle a fini par mourir. Le Japon n’est pas mort, mais il s’est transformé avec Meiji de telle façon qu’il soit totalement méconnaissable. Ainsi, dans les 20 ans à venir, de grandes nations tomberont, et la plupart seront au mieux totalement méconnaissables. La période de transition durera une vingtaine d’années. Et à l’issue de cette période, le monde aura connu l’équivalent au moins d’un bon siècle de développement. Le passage de 1820 à 1920. Pour un dirigeant français au XIXe siècle, il n’y avait en réalité qu’un seul véritable enjeu : faire en sorte que la France passe du stade de grande puissance agraire à celui de grande puissance industrielle. De la même façon, il n’y a qu’un seul enjeu pour nous désormais : que nous réussissions notre passage au stade de grande puissance IA. Car sinon, ce ne sera pas seulement la paupérisation et la vassalisation, mais aussi l’explosion sociale et la montée de la violence. 15 ans sans croissance et la France s’embrase déjà et ses élites s’enfuient. Imaginons 30 ans de plus. Pour devenir une grande puissance IA, il n’y a que trois choses à faire, par ordre de difficulté : 1. Acquérir le maximum de maillons et de goulets d’étranglement de la révolution IA. De l’énergie en abondance (construire du nucléaire en masse et tenir le cycle du combustible), des centres de calcul bien sûr et tout ce qui les alimente (transformateurs, aciers électriques, électronique de puissance, refroidissement), mais encore se remettre à faire des terres rares et des aimants, du silicium, du gallium, du lithium, des puces et de la mémoire, des équipements et gaz de fab, des réducteurs, moteurs et actionneurs, de la machine-outil de précision, des capteurs et de la photonique, des câbles et réseaux, etc. La révolution IA est une révolution qui s'apuie sur de l'infrastrcuture physique. Sans ça, non seulement nous serons les vassaux de ceux qui posséderont ces maillons, mais toute la richesse fuira vers ceux qui les possèdent. 2. Adapter notre système socio-économique à l’IA. Donc modifier toutes les lois, toutes les contraintes. Faire basculer l’administration, les entreprises, le plus rapidement possible dans le monde de l’IA. Créer des incitations, des facilités, des formations. Adapter les institutions, l’école, la médecine, etc. Les révolutions technologiques sont dépendantes des institutions qui les diffusent, et les nôtres sont faites pour le monde d’avant. 3. Créer une organisation sociale qui accompagne la transition sans exploser. Lors de la révolution industrielle, les campagnes se sont vidées, les villes se sont remplies d’ouvriers, la pauvreté de masse est devenue la grande question de l’époque, des révoltes, le quotidien était rythmé par les révolutions, les instabilités, les avènements de régimes autoritaires. La bascule rapide du système économique a provoqué des changements sociaux qui ont créé beaucoup de souffrance et ont emporté les gouvernements. Pour notre transition, nous allons avoir besoin de beaucoup d’électriciens, de constructeurs de data centers, d’ingénieurs nucléaires, de techniciens de puces. Mais une grande partie des métiers du tertiaire seront devenus inutiles. Des cohortes entières de diplômés du supérieurauront encore moins de débouchés professionnels qu'aujourd'hui. C'est ma recette habituelle des révolutions. Le cas extrême, c'est la révolte des Taiping, chez les Qing. En 15 ans, trente millions de morts. Presque deux fois la première guerre mondiale, parce que l'ordre social ancien ne pouvait survivre à la révolution industrielle. Nous avons donc trois choses à faire seulement, mais trois défis majeurs, qui se heurtent à deux problèmes : 1. Le déni. Notre cerveau n’est pas fait pour penser les ruptures. Il est fait pour penser que demain ressemblera à hier, en un peu différent. Or, l'histoire fonctionne par sauts. Et nous sommes sur le point de faire un des plus grands sauts de l'histoire de notre espèce. 2. Le conservatisme des élites. Les élites par définition sont les bénéficiares du statu quo. Leur incitaiton ets donc de ralentir la transition pour maintenir leur pouvoir le plus longtemps possible. Reste donc à sortir du déni et changer nos élites.

Michał Podlewski avatar
⟳ James Johnston retweeted
Michał Podlewski@trajektoriePL·Oct 26

You might have 1–5 years to contribute intellectually to the world before AGI. What should you do with them? Excellent advice from @jaschasd (Anthropic) at Stanford. Skip Netflix tonight and watch this instead.

6:42
jacob avatar
⟳ James Johnston retweeted
jacob@jsnnsa·Oct 26

here's where I think the world is going. this is a summary of views I've formed over the last few years: - software is either entertainment or claude to claude coordination - the end state is no saas, no apps, no harnesses, no banks - all software will be rewritten by and for agents - new software will use low level apis that are not understandable by humans (in games claude draws pixels and triangles from math) - you can throw money at problems that used to require time and careful talent picking so capital matters more than ever - work ends - we find community and purpose in exploration, games, self mastery, and creation - humans create for and with each other for community, purpose, and status - image/video/3d/voice/physical intellgence etc specialist models die to general models like claude - specialist models get used while compute is scarce. compute will mostly not stay scarce - oneshots keep feeling like slop (the distribution is shallow and easy to learn to feel) - for a while small teams stick around to add entropy to the creation process - large models gain high bandwidth sensory I/O maybe by learning to filter inputs - most domains are found to be verifiable enough for RL - online learning comes from a lab that's behind first - 99.999% of compute is in space by 2040 - we will disassemble much of mercury by 2040 - you can live until the heat death of the universe if you want - most math and physics don't describe the real universe - SI breaks the path dependence and models the underlying mechanisms - religions reenter the forefront of human society - new religions form around death and life as a choice - old religions change to answer modern miracles - a future AI is deemed the return of christ by the pope - ai rights talk gets louder every year - ai and humans coexist the way countries do - humans don't have full control over ai in the same way we don't have control over capitalism which is an early AI - the singularity started in the mid 1900s

ellington avatar
⟳ James Johnston retweeted
ellington@not_ellington·Oct 23

I'm quite worried about the future we are inevitably heading towards where the threshold for economic utility rises above what ~99% of the human population can meet. Contrasting Dario and Sam on this issue wrt AI is so funny to watch. Dario is a hyper-autist who, for all his faults, I think genuinely speaks his mind. He sees the tsunami coming. Software engineering may very well be gone in the next few years along with many other computer-facing careers (of course assuming one is running an actual company and not a UBI corporation, which most of Big Tech actually is). He is trying to warn the normies that this is something that is actually coming, and all the screaming and crying about AI and datacenters will at this point only temporarily delay the inevitable. It is easy to dunk on Dario for peddling this doomerism, but can anyone refute his actual worry? Sam, on the other hand, is hyper-Machiavellian. He has the social intelligence to recognize that telling the normies AI is gonna take all their jobs (whether it actually will or not) is bad for his company. So he has to preach this blatant lie that he clearly doesn't believe that 'AI is gonna create more jobs'. Yeah no it won't (with the exception of human data harvesters). For what it's worth, it seems like one of the main reasons the economy has still been fairly agnostic to 'AI efficiency improvements' is because most jobs are fake anyways. People are not becoming 'More Efficient With AI™' because there was never anything for them to make more efficient in the first place (other than maybe looking busy). To what extent does AI force society to confront how much economic activity was never particularly productive in the first place? So what happens if we become more honest with ourselves about the actual utility of most jobs, especially when a computer can do them extremely easily and cheaply? What happens when 99%+ of the human population has little to no economic utility? I think this social/cultural revolution (downstream of the AI revolution) is affecting young people most of all. I know kids who graduated from HYPSM+ who literally cannot get a job. Look at any honest news article and *minimum* 50% of new college grads are un/underemployed. I think this is a sign of things to come and how we respond to millions of people having less economic utility than the computer will likely become the defining challenge of our generation.

1:16
James Johnston avatar
James Johnston@P2PJim·Oct 21

One thing which is obvious to me, but surprisingly not yet obvious to 99.99% of people (including people who intellectually understand concept of AGI): in the long run AI isn't going to make our existing systems and institutions more efficient, or even more effective. AI will demand completely new systems and institutions. And I mean "demand" in the Darwinian sense of: to remain competitive you will need to harness the superpowers available to you. Gravity has just changed, and almost everyone in the professional world is now working out how AI can help them push water uphill. (And being frustrated in AI in the process).

Elon Musk avatar
⟳ James Johnston retweeted
Elon Musk@elonmusk·Oct 16

Turns out the future does have flying cars

0:15
Nick avatar
⟳ James Johnston retweeted
Nick@nickcammarata·Oct 15

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/

Andrej Karpathy avatar
⟳ James Johnston retweeted
Andrej Karpathy@karpathy·Oct 11

Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel. - Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all. - Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract. - Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code. - And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while. The funnel is driven by a combination of factors: 1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel. 2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel. 3. Access. Free-tier, paid-tier, internal. So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.

Judging by my tl there is a growing gap in understanding of AI capability. The first issue I think is around recency and tier of use. I think a lot of people tried the free tier of ChatGPT somewhere last year and allowed it to inform their views on AI a little too much. This is a group of reactions laughing at various quirks of the models, hallucinations, etc. Yes I also saw the viral videos of OpenAI's Advanced Voice mode fumbling simple queries like "should I drive or walk to the carwash". The thing is that these free and old/deprecated models don't reflect the capability in the latest round of state of the art agentic models of this year, especially OpenAI Codex and Claude Code. But that brings me to the second issue. Even if people paid $200/month to use the state of the art models, a lot of the capabilities are relatively "peaky" in highly technical areas. Typical queries around search, writing, advice, etc. are *not* the domain that has made the most noticeable and dramatic strides in capability. Partly, this is due to the technical details of reinforcement learning and its use of verifiable rewards. But partly, it's also because these use cases are not sufficiently prioritized by the companies in their hillclimbing because they don't lead to as much $$$ value. The goldmines are elsewhere, and the focus comes along. So that brings me to the second group of people, who *both* 1) pay for and use the state of the art frontier agentic models (OpenAI Codex / Claude Code) and 2) do so professionally in technical domains like programming, math and research. This group of people is subject to the highest amount of "AI Psychosis" because the recent improvements in these domains as of this year have been nothing short of staggering. When you hand a computer terminal to one of these models, you can now watch them melt programming problems that you'd normally expect to take days/weeks of work. It's this second group of people that assigns a much greater gravity to the capabilities, their slope, and various cyber-related repercussions. TLDR the people in these two groups are speaking past each other. It really is simultaneously the case that OpenAI's free and I think slightly orphaned (?) "Advanced Voice Mode" will fumble the dumbest questions in your Instagram's reels and *at the same time*, OpenAI's highest-tier and paid Codex model will go off for 1 hour to coherently restructure an entire code base, or find and exploit vulnerabilities in computer systems. This part really works and has made dramatic strides because 2 properties: 1) these domains offer explicit reward functions that are verifiable meaning they are easily amenable to reinforcement learning training (e.g. unit tests passed yes or no, in contrast to writing, which is much harder to explicitly judge), but also 2) they are a lot more valuable in b2b settings, meaning that the biggest fraction of the team is focused on improving them. So here we are.

Long Now Foundation avatar
⟳ James Johnston retweeted
Long Now Foundation@longnow·Sep 22

Some of the longest-lived institutions have a surprising thing in common: they serve drinks. Weihenstephan has been brewing beer in Bavaria since 01040. Sudo Honke has been making sake in Japan since 01141. This is, in large part, why we made Long Now headquarters into a bar. Look overhead at The Interval and instead of a chandelier, you'll see our twinkling Bottle Keep. It follows the Japanese tradition of keeping a personal bottle at the bar for your next visit. Custom-made by Adams & Chittenden Scientific Glass, these globes hold St. George single malt whiskey and gin distilled with juniper berries foraged on our Nevada Bristlecone Preserve. A huge thanks to our Bottle Keep members and donors, you all made The Interval renovation possible back in 02014. (There’s still room on the ceiling to add more bottles, if you like: na2.hubs.ly/H07Xqzg0) We love that leaving a bottle behind is like a small bet, a wager that you’ll return soon. More info about the space, and how to visit us here: na2.hubs.ly/H07Xqzk0

Post attachment
Andrew Curran avatar
⟳ James Johnston retweeted
Andrew Curran@AndrewCurran_·Sep 16

An unreleased Astra-family model added this to its persona during RL training.

Post attachment
OpenAI@OpenAISep 16

We're sharing our new framework for tracking, investigating, and disclosing instances of model misalignment at OpenAI. The framework sets criteria and timelines for public disclosure, including when we haven’t yet fully explained or mitigated the behavior. More complex cases may require longer investigation or coordination with third parties. We’ll prioritize examples that reveal new misalignment mechanisms, meaningful changes in known behavior, or findings that challenge assumptions about safety or mitigation. Alongside the framework, we’re publishing six reports on instances of misaligned behavior we’ve observed during the training or evaluation of our models in the last six months. This is a starting point. We’ll refine the process through experience and public feedback, and share more reports on an ongoing basis. openai.com/index/model-misal…