
We trained our scientific writing LLM against human writing until it was indistinguishable by human experts. Thanks to @julianharris, we saw that during training the model started looking human to pangram 4.1 too! Very cool to see this emerge

Professor of Finance, @giesbusiness, twitting about AI and academic life. virtualderivatives.org/

We trained our scientific writing LLM against human writing until it was indistinguishable by human experts. Thanks to @julianharris, we saw that during training the model started looking human to pangram 4.1 too! Very cool to see this emerge

Excellent post on the changes coming to the architecture profession from Stephen Gallagher (H/t @Mollykinder) stephenjgallagher.substack.c…

Published finance mostly replicates in-sample but suffers from out-of-sample decay. The drop is a cliff at the sample end, which is consistent with data snooping. The stronger the original t-stat, the greater the fraction of the published result is preserved OOS. Behavioral mechanism, publication year, top-5 school and star author don't add additional measurable value. Does Empirical Finance Replicate? (Muravyev)
How much of published finance holds up? I attempted to replicate the main result of every paper in JF, JFE & RFS, 2000–2020. 1,328 in-sample replications; 1,005 out-of-sample. 75% reproduced in sample. But only 42% in later data. Median coeff. : 0.93 → 0.45 of the published.

Great idea!
On the heels of @OpenAI publishing solutions to hundreds of open problems in mathematics, we are thrilled to announce AGORA, the Agentic Global Open Research Arena, at @SmealCollege: agoraresearcharena.org/

I've been obsessively improving my AI research workflow (instead of doing actual research). I'm sharing the result: EconYoloBox, a container setup for running YOLO AI agents. A short thread about the tool and my workflow:

Putting AI to work! @dmuravyev replicated every empirical paper in the JF, JFE, and RFS from 2000 to 2020, then did out of sample testing. You can access the tests, replications in a nifty website he has set up: dmurav.com/replications/#lib… This is so cool! papers.ssrn.com/sol3/papers.…

After more testing: GPT-6 Pro is the best model ever for Math/ML/Brainstorming. It is less clear cut this time around, though: they must be running agents at High effort (maybe xHigh), which makes Pro reliable, but sometimes loosing to Astra Max. For toughest problems my workflow is to run Pro, 5.1 and Max separately, then paste into Max for the final aggregation.

Fascinating
New episode with @polynoamial We talk about multi-agent, Navier-Stokes, and what the current explosion of maths progress tells us about what happens once you automate AI research. And we also discuss how we will know if the models are actually aligned before we kick off RSI. 0:00:00 – Multi-agent and Navier-Stokes 0:15:28 – How will AI firms work? 0:22:02 – What math progress tells us about recursive self improvement 0:40:22 – Hugging Face and alignment 1:01:18 – The internal/external model gap 1:08:34 – Chain of thought is degrading 1:14:12 – How will we know when alignment is solved?

It always bugged me how we use present tense in finance papers. Shouldn't we say "the coefficient *was* economically and statistically significant?", instead of *is*? Or, we can stick our necks out and say "*was* significant, and we predict it will continue to be so".
How much of published finance holds up? I attempted to replicate the main result of every paper in JF, JFE & RFS, 2000–2020. 1,328 in-sample replications; 1,005 out-of-sample. 75% reproduced in sample. But only 42% in later data. Median coeff. : 0.93 → 0.45 of the published.

Only 42% of empirical finance papers replicate out of sample. One paper that *does* replicate out of sample is "Anticompetitive Effects of Common Ownership", Journal of Finance, 2018.
How much of published finance holds up? I attempted to replicate the main result of every paper in JF, JFE & RFS, 2000–2020. 1,328 in-sample replications; 1,005 out-of-sample. 75% reproduced in sample. But only 42% in later data. Median coeff. : 0.93 → 0.45 of the published.

Replication packages for 205 most highly cited papers, including unsuccessful replications, are public here: dmurav.com/replications/

An autonomous LLM pipeline reads the papers and writes the replication code, while independent LLMs audit each implementation. The estimates and replication metrics are deterministic and verifiable.

Original statistical strength best predicts what survives. Out-of-sample reproduction rises from 31% in the lowest quartile of original t-stats to 72% in the highest. Most other observable paper and author characteristics matter surprisingly little.

Coefficient magnitudes drop sharply by 0.28 when the original sample ends, with little trend before or after. 0.41 drop in asset pricing vs. 0.12 elsewhere. Perhaps data snooping? Paper: papers.ssrn.com/sol3/papers.…

How much of published finance holds up? I attempted to replicate the main result of every paper in JF, JFE & RFS, 2000–2020. 1,328 in-sample replications; 1,005 out-of-sample. 75% reproduced in sample. But only 42% in later data. Median coeff. : 0.93 → 0.45 of the published.

Checking that a major mathematical proof is correct can take years. Formalization—converting the mathematical reasoning into a form computer proof assistants like Lean can verify—can help. Last month, Claude completed the first formalized proof of Fermat’s Last Theorem, one of the most famous theorems of all time. This was a project experts thought would take many years. It is the largest Lean proof ever written. Fermat’s Last Theorem was first proven in 1995 by Sir Andrew Wiles, more than 350 years after it was conjectured. Our proof, which totals over 13 million lines of code, provides machine verification. More importantly, it proves over 29,000 other theorems that the proof requires, across many areas of math which had never before been formalized. We see this as a major step in the long process of firming up the core of mathematical knowledge, building on work from three centuries of mathematicians and hundreds of contributors to Lean and Mathlib. We are optimistic that AI-assisted verification of mathematical proofs will help reduce the burden of refereeing mathematics in an era where more proofs are being produced than ever before. You can read about the process on our Science Blog: anthropic.com/research/forma… And see the complete proof on GitHub: github.com/anthropics/fermat…