←
FleetingBits12,274 posts
FleetingBits avatar

FleetingBits

@fleetingbits

sf thinkcat

◷ Joined September 2023▧ 2,220 Photos and videos
Dodo avatar
⟳ FleetingBits retweeted
Dodo@Mechanical_Dodo·Nov 16

the Norvisphere marches on through the shallow sea, scooping up as many knowledges as possible: to compare and trade; to cherish and enjoy; and to brandish in confrontation with unknowable fate on the final shore

norvid soon: I have returned.

Post attachment
FleetingBits avatar
FleetingBits@fleetingbits·Nov 15

some quick thoughts on ai & value creation 1) it may be obvious at this point, but it is worth noticing that economic value from ai accrues to frontier labs and not to application ai companies 2) frontier labs have created ~$2.5tn of market value since 2023 while application ai companies have created ~$360bn and this includes companies like elevenlabs 3) in addition, openai and anthropic have ~$130bn of revenue while application companies have ~$13bn in total revenue; > $115bn of frontier revenue doesn't even pass through application ai companies 4) i think that this emphasizes the fact that most value from ai does not come from productitization because if it did we would expect to see more revenue at application companies 5) instead, the value seems to come from the model itself and then from integrating those models effectively into existing organizations; but, the later is not a product feature movement 6) there have been signs of this since at least 2023, when it was so easy for large companies to introduce ai variants of their existing products, it was just an api call 7) but, i think we can more definitively say today that the model is the product and the scaffolding that you build around them is much less important on the whole 8) and, this is very good for the frontier labs, which may become less organizationally nimble, but which will retain a very strong advantage in model training itself 9) due to high capital requirements for the necessary compute, an existing lead in research agents, etc... 10) and, frontier labs will therefore be likely to both create and capture most of the concentrated value with respect to ai, across a very wide range of verticals

FleetingBits avatar
FleetingBits@fleetingbits·Nov 11

it was pretty obvious, but i have been saying for a while that frontier labs would setup their training such as to not release models with frontier ai r&d capabilities to the public

Post attachment

some thoughts on gpt-rosalind 1) gpt-rosalind is a new openai model focused on biology and drug discovery 2) it's available through openai's trusted access program; early partners include moderna, amgen and the allen institute 3) openai has not published a system card for gpt-rosalind, but i expect it would score higher on biorisk benchmarks than gpt-5.4, given that it is stronger on chemistry and experimental design and analysis 4) i expect this or some variant to be the future of model releases; dual use capabilities only being made available to trusted partners; we saw an example of this already with claude mythos 5) i also expect that as dual use capabilities become more extreme, access will narrow to a smaller set of trusted partners, most likely those with significant institutional and, in certain cases, government backing 6) it's interesting that openai built a separate bio model rather than train a general model with safeguards removable for trusted users; this may be their approach to cyber capabilities as well 7) claude mythos was notable because its cyber capabilities were described as emergent; they came from better code understanding and autonomy rather than offensive cyber training 8) but you can imagine a version of claude mythos or gpt-5.4 post-trained on offensive cyber skills and released separately to trusted partners 9) it's also possible the future pipeline looks more like a central general model developed internally and never released 10) with frontier labs distilling capabilities from this central model so that dual use capabilities are only available to trusted partners 11) and some skills, like machine learning research, are retained inside the lab entirely and not provided to customers 12) anyway, i'm not wedded to this approach, but labs do need to figure out how to release models to the public without certain capabilities; and, monitoring outputs alone may not be enough 13) also importantly, the failure to release a system card for gpt-rosalind is a real omission, which openai should explain 14) as system cards are important for both policy makers and the public to understand the risks of new model releases

FleetingBits avatar
FleetingBits@fleetingbits·Nov 8

some quick thoughts on frontier lab secrecy 1) sam mccandlish made a post where he said that anthropic would share more about rsi in the coming weeks; but otherwise said very little 2) i guess the point of the post was to control perception of anthropic while they wait to come out with their own version of the metr rsi report; 3) since otherwise, he doesn't really say whether he agrees with it, whether he disagrees with it; he says the reductions were justified, but not much as to why 4) something to note is that frontier labs are extremely secretive; and, this is a little strange given how often researchers move between them 5) your most important competitors probably have a reasonable picture of your research efforts, the general pace of progress, how your teams are organized, etc... 6) but, the general public has very little of this context; and, talking to researchers can have a surreal feeling when they treat like obvious stuff as secret 7) part of the explanation is that the specific details of how experiments are implemented matter; knowing the broad direction is not enough to replicate everything 8) so, there are important things to protect; but you then have to organize hundreds of researchers such that they know what they can and cannot say 9) and, i guess it is very hard to expect everyone to have good judgment, exercise it well, model properly what a competitor or other researcher might know 10) so, the easiest rule to communicate and expect compliance with seems to be to "keep everything secret, no matter what it is" and this is the rule 11) this makes sense at a corporate level, but leaves the public with little context for evaluating claims; and, the labs then either make statements or selectively release stuff to influence opinion 12) i don't really blame them; but, it makes commenting on safety, research acceleration, etc... quite difficult

I'm Sam, one of Anthropic's co-founders, and I lead much of our safety work. I'm going to have more to share on that work over the coming months. I thought I'd start by sharing a few thoughts on the latest report from @METR_Evals. The speed of progress in model capabilities is one of the most important questions in AI safety, and METR’s work here provides the public with an independent, external view into how this is happening inside Anthropic. I’ll talk more about some of the details below, but first some context for those new to this work. How quickly AI progress is accelerating has many important safety implications, since it determines how quickly the risks from advanced models increase. But, because most research happens within the frontier labs, it’s difficult for external observers and policy makers to get a strong sense of what that rate is. We kicked off the process with METR that led to this report earlier in the year, in anticipation of progress continuing to accelerate. For this report, we gave METR access to a range of sensitive internal data, including benchmark scores for unreleased models, estimates of how models are progressing, and interviews with our researchers. They had complete independence to form their own conclusions. Many of their conclusions match my own. AI is speeding up our research, but it’s hard to say exactly how much. While they conclude AI can't yet make the most important contributions to model training on its own, they think the pace of development could, by default, accelerate further. This work began well before and is separate from the embedded third-party evaluators that are part of our commitment to pacing the frontier, but it’s a good illustration of why we think evaluators are a good idea. The more access reviewers have, the clearer a picture they can form of the situation and the better-positioned they are to highlight situations like this one. We redacted a lot, and I accept that this makes it harder for others to check METR’s work. But we determined that sharing exactly what is driving the speedup isn’t the right thing to do, both because it’s commercially sensitive and because it risks handing out a roadmap for others to go faster. That would make the risks METR is worried about worse. That’s a tradeoff. I think we made the right call, though we’re continuing to assess the pros and cons here. In the coming weeks, we plan to share a report with our own analysis of the speed of AI progress within Anthropic, including where we stand against our Responsible Scaling Policy, plus new steps we’re taking to strengthen our safety and security practices as AI keeps improving quickly.

FleetingBits avatar
FleetingBits@fleetingbits·Nov 6

i've been working with claude on a programming game, inspired by screeps, for my personal website, fleetingbits.io; there are two factions, one based on scheme and the other on apl

1:08
FleetingBits avatar
FleetingBits@fleetingbits·Nov 5

some quick thoughts on metr's evaluation 1) metr reports that anthropic’s models are improving 1.9x as fast as they were before october 2025 2) the main evidence in the report comes in the form of an expanded version of anthropic's internal eci and reports from researchers 3) metr also says that they have other evidence, which anthropic redacted; odds are that this is information about algorithmic efficiency over time 4) the anthropic eci information looks convincing to me; it looks like anthropic is no longer releasing its best models and this has made public eci less useful 5) when you add in historical models of other providers to the eci, the speedup falls to 1.4x, since anthropic models were weaker than the frontier models of other providers (e.g. openai) earlier in 2025 6) researchers reported that they are able to get a lot of uplift from research agents; they can launch more experiments; less handholding of experiments 7) this makes sense, the uplift appears to have began sometime around opus-4.5, which appears to have began to automate coding for many programmers 8) this just emphasizes how important it is for anthropic and other labs to release their internal eci on a regular basis so observers have better information 9) also, they should release more information on their research agent spend; so far, openai has been the most transparent on this topic (~33 day doubling time) 10) i think that more uniformity of reporting would do a lot of good; regular internal eci reporting, standardized research agent reporting 11) since, it would make the public discourse much more regular, well-informed, there would be much less alarmism 12) note, i think that a lot of people will overreact to this because they view acceleration as the start of the end of the world; but this says little about alignment 13) by the way, i do wonder if model 2 is just mythos 5.1 with additional post-training for ai r&d; it would make sense for the labs to attempt to separate this out 14) since, it would make those capabilities very hard to impossible to distill and it would presumably also make classifiers less important / upset users less

1. Benchmarks provide one piece of evidence for acceleration. Since Oct 2025, AECI scores for Anthropic’s models have grown at 1.9x [95%: 1.5–2.8] the earlier rate. Analysis choices affect this estimate, and it remains unclear how benchmark scores map to real-world capabilities.

Post attachment