Report says Anthropic's AI is helping it build new models twice as fast

Anthropic’s AI models are improving roughly twice as fast as they were a year ago, and most of that speedup now comes from AI itself, according to an independent assessment by the AI safety nonprofit METR published on Thursday, November 5. The findings come from an 11-week investigation inside the company, but Anthropic redacted large parts of the evidence before publication, which has left researchers both alarmed by the pace and uneasy about what remains hidden.

The report lands at a sensitive moment. Anthropic is planning to go public later this month in what could be the biggest initial public offering of stock ever, while its CEO, Dario Amodei, has publicly urged the industry to slow down.

What METR found inside Anthropic

METR (Model Evaluation and Threat Research) is a Berkeley, California-based nonprofit that has evaluated model capabilities for Anthropic, OpenAI and other labs for years. It is funded through donations and does not accept compensation for its work, although the labs it partners with provide access and tokens that it uses for evaluations, research and engineering.

Anthropic invited METR to review its internal data and interview staff. The analysis drew on data provided over about 11 weeks, from mid-August through late October, plus employee interviews, and focused on whether the growth rate of Anthropic’s internal capabilities had accelerated and how much of that was attributable to AI. METR said it was given “unprecedented access to internal data”.

The headline estimate: Anthropic is improving the capabilities of its models to do “well specified tasks” around twice as fast as it was a year earlier. In METR’s words, “Anthropic appears to have made as much progress on benchmarks in the past year as they did between October 2023 and October 2025”. The Verge reports METR’s framing that benchmark scores suggest “that Anthropic has made roughly two years of historical progress over the last year”. METR also noted internal disagreements and uncertainty within its own team, but said it believed the company’s rate of improvement had doubled.

METR published two documents: a holistic assessment of AI R&D acceleration at Anthropic, and a separate review of the “Risks from automated R&D” section of Anthropic’s August 2026 Risk Report, written by the same METR staff.

How much of the speedup comes from AI

The central question is why progress is faster. Historically, scaling up training compute (the computing power used to train models) has been a major driver of AI progress. METR said it was able to “confidently” rule out growth in training compute as the reason for this acceleration. Instead, its researchers said the majority of the acceleration as of October “is due to AI,” with AI’s contribution growing over time.

Informal Slack polls and interviews with Anthropic staff pointed the same way. According to METR, employees using Anthropic’s most capable internal model self-reported being around 12 times as productive as they would have been without AI assistance, around double what staff reported a few months earlier while using Mythos 5. These are self-reports, not measured output.

For now, most of the gain appears to come from AI assisting human researchers rather than working independently. The report states: “We understand that AI is not yet able to make highly significant and fully autonomous contributions to the most important parts of the model training process”. But Anthropic researchers described significant “signs of life” implying that could happen soon, METR said, and METR warned that “there are some signs that fully autonomous contributions will play a larger role soon”.

What worries researchers most is compounding. METR said a key concern is that using AI to improve AI could start to compound the acceleration, especially since it found no “bottlenecks,” such as limits on compute or data, holding back Anthropic’s progress. The Verge reports METR found no clear signs of diminishing returns that could prevent “even faster acceleration”.

Did Anthropic cross its own warning line?

Anthropic’s Responsible Scaling Policy (RSP, its internal rulebook for handling dangerous capabilities), version 3.4, sets two triggers for “Automated R&D in key domains”: a model that can “fully substitute for our entire set of Research Scientists and Research Engineers, at competitive costs (i.e., within a factor of 5),” or a “dramatic acceleration” of AI progress likely related to automating AI R&D, which METR calls the 2x Acceleration Threshold. Anthropic’s August Risk Report argued neither had been crossed and described the 2x threshold as “a potential early warning, rather than evidence that the threat has already materialized”.

METR agrees that models deployed internally as of July 15 posed low risk of directly automating R&D in key domains or causing acute catastrophic risks. However, it says Anthropic’s claim that the early warning threshold had not been crossed “relies on contestable judgment calls,” that it is plausible the threshold had been crossed, and that the public Risk Report “overall understates the level of acceleration that was occurring”.

Three unreleased models behind the numbers

Much of the acceleration and productivity METR describes is attributable to models the public cannot use. Over the past six months, Anthropic has finished training three models it has not released, codenamed Model 2, Model 3 and Model 4, which completed training in July, August-September and September-October respectively. METR also identified an unreleased model from 2022, called Unreleased 2022. “These are the codenames shared with us; we understand they have different internal codenames,” the researchers wrote. None of the sources give release plans for these models.

What Anthropic’s own measurements show

Anthropic has published its own figures on the same question. It describes a prototype Anthropic R&D Automation Index, built by cataloguing every kind of AI R&D work at the company, rating how automated each task is, and aggregating the ratings. It uses an automation scale developed by Epoch AI that runs from AL0 (no AI involvement) to AL5 (AI fully autonomous, no human in the loop); at AL3 AI “collaborates” under close human direction, and at AL4 AI “leads,” completing most of a task end-to-end from a high-level prompt while a human supervises.

Measure	Figure	Who says so
Pace of capability gains vs a year earlier	About 2x, on well-specified tasks	METR
Self-reported productivity with top internal model	About 12x	METR, from staff polls
Share of AI R&D work Claude “leads” (August 2026)	26%	Anthropic
Share at or above “AI collaborates”	Above 90%	Anthropic
Fully autonomous work in any measured subset	None	Anthropic
Unreleased models finished since July	Three	METR

Anthropic acknowledges a weakness in its own method: it uses its own models to evaluate its systems, so the “judge” model could make the same kinds of errors as the model it is checking. It suggests third parties or other developers’ models could verify such measurements, and that these numbers could trigger stronger requirements, such as a fixed testing window before a new model is used for further AI R&D. The company also says it plans to embed independent evaluators from multiple organizations with access to internal processes, systems and data comparable to its internal risk teams.

Redactions and the limits of the review

Under its agreement with METR, Anthropic could redact any non-public information; METR in turn kept the right to disclose whether redactions were used and whether its findings depended on them. Anthropic used that right extensively. “We submitted a ~100-page full report on our assessment to Anthropic. For the public version, Anthropic redacted a majority of the evidence we found most informative,” METR wrote on X. The redactions limited how METR could communicate more detailed estimates of acceleration.

METR also credits Anthropic for openness: the lab “shared more information than was initially scoped for this project, surpassing our expectations”. But The Verge notes that METR did not get unrestricted access, nor the “employee-like” access Anthropic promised other METR safety researchers under a separate program announced last month.

Anthropic co-founder Sam McCandlish, who is also its responsible scaling officer, responded in a long post on X saying many of METR’s conclusions match his own. “AI is speeding up our research, but it’s hard to say exactly how much,” he wrote. He acknowledged that redactions make it harder for others to “check METR’s work,” but said sharing “exactly what is driving the speedup isn’t the right thing to do,” because it is “commercially sensitive” and “risks handing out a road map for others to go faster”.

The report itself carries caveats. METR said it could be overstating its findings because the acceleration might be limited to “well-specified tasks” that can be graded automatically or by a language model, rather than “messy” tasks that are harder to define. It is also possible Anthropic understated its earlier progress, making the latest figure look bigger.

Why the timing matters: slowdown calls and an IPO

The assessment overlapped with Amodei’s push for restraint. In a September 12 essay he urged the industry to “slow the pace at which we improve the capabilities of AI models”. He wrote that AI advances have been driven primarily by recursive self-improvement, which he defined as “AI’s growing ability to build the next generation of AI,” adding: “Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all”. In the same essay he backed embedding external evaluators in leading labs and named METR as an appropriate one; OpenAI CEO Sam Altman said at the time that he also supported independent evaluators with employee-like access.

The Associated Press notes the findings matter because details about frontier AI and its risks have mostly come from the companies themselves, which President Donald Trump’s administration has encouraged to police themselves on safety. McCandlish made a similar point: “because most research happens within the frontier labs, it’s difficult for external observers and policymakers to get a strong sense of what that rate is”.

What it means for builders

• The public models lag the internal ones. Much of the reported speedup comes from unreleased models, and three have finished training since July. Builders working with Anthropic’s API are using systems behind what the company runs internally.
• The gains are clearest on checkable work. METR’s estimate covers “well-specified tasks” that can be graded automatically, the same kind of tightly scoped, testable tasks many coding and agent products depend on. Whether the speedup extends to “messy” work is an open question METR itself raises.
• Agent oversight is becoming a measured problem. Anthropic says its staff increasingly delegate long-running work to agents that hand tasks to each other, and that while individual agents rarely misbehave in its monitoring, rare events can become regular at scale. Teams building multi-agent systems face a version of the same issue.
• Treat the numbers as estimates. The 12x productivity figure is self-reported, and the most informative evidence is redacted.

What to watch next

Several threads are open. Anthropic’s planned IPO later this month will bring more public scrutiny of how it describes its own pace. The separate program giving outside evaluators “employee-like” access, and Anthropic’s plan to embed third-party evaluators, will show whether future reviews face the same redaction limits. Watch for whether Anthropic publishes its R&D Automation Index regularly with a public methodology, as it says any frontier developer could. And METR’s warning that fully autonomous contributions may soon play a larger role sets up the key question for the next review: whether AI moves from helping Anthropic’s researchers to doing the most important parts of model training on its own.