Mistral Large 4 ties OpenAI's GPT-6.1 Terra on a coding agent index

Two days after Mistral released the open weights for Mistral Large 4, the model’s standing on independent coding rankings is coming into focus. Coverage of Artificial Analysis’ Coding Agent Index puts Large 4, nicknamed “Le Chonk”, at a score of 48, the same as OpenAI’s GPT-6.1 Terra and Codex. That places a French open-weight model level with a closed OpenAI coding pair on a test of multi-step software work, though the outlets do not agree on whether it is the single best open model on the index.

The last Unicorn article covered the weights release and noted that most benchmark figures then came from Mistral itself. This update focuses on the third-party coding ranking, what it measures, how strong the evidence is, and what it means for teams picking a model.

What the Coding Agent Index result shows

ZDNET reports that on Artificial Analysis’ Coding Agent Index, which combines a model’s scores across three benchmarks into one composite, Large 4 scores 48, and that “Codex GPT-6.1 Terra also scores a 48 overall”. Shattered, citing reports from Gigazine and The Decoder, gives the same figure and says it matches GPT-6.1 Terra and Codex; it describes the two OpenAI entries as a “closed pair”. ZDNET’s wording treats them as a single entry, “Codex GPT-6.1 Terra”. Either way, the reported score is the same.

The index tests agentic work: multi-step tasks in which a model reads a repository, edits files and runs commands, rather than completing a single snippet of code. Shattered argues that a score of 48 shows the model holds up across those loops.

Two caveats come with the number. First, Shattered says it could not pull the Artificial Analysis leaderboard directly and asks readers to “treat the exact rank order as reported rather than independently checked”. Second, ZDNET reminds readers that “benchmark scores themselves should be taken with a grain of salt, especially considering how many models cheat”. Gigazine’s summary, as relayed by Shattered, also notes that Mistral paired the model with its Vibe coding tooling in the comparison, so the result reflects the model plus Mistral’s agent setup.

Where the sources disagree on Large 4’s rank among open models

The headline tie with OpenAI is consistent across sources. The ranking among open models is not.

• ZDNET writes that the model ranks “at the top of Artificial Analysis’ Coding Agent Index among open models globally”.
• In the same article, ZDNET reports that on the overall index “ML4 is a point behind GLM-5.3, scoring a 48 to GLM-5.3’s 49”. GLM-5.3 is an open model, so those two statements do not sit easily together.
• On Terminal-Bench 4.0, an agentic test of how well a model drives a command line, ZDNET says Large 4’s 41% beats GLM-5.3 “by one small but important point”, while SiliconANGLE, as cited by Shattered, says the result ties GLM-5.3.

The safest reading from these sources: Large 4 sits in the top group of open models for coding, at or within a point of GLM-5.3 depending on the measure, and level with GPT-6.1 Terra on the composite index. Shattered also cautions that the reported ties come from indexes that favour token-efficient agents, and that whether the gap to the frontier has fully closed “depends on which benchmark you trust”. Shattered further reports that the model still trails the best Chinese open models on overall intelligence scores.

How Large 4 changed between preview and open weights

The preview launched on October 6. Shattered counts 23 days between that and the October 29 weights release, a window in which Mistral says it fed preview feedback back into training and kept scaling reinforcement learning, the stage of training in which a model is rewarded for successful outcomes.

The clearest change is on Terminal-Bench 4.0, where the score rose from 28% in preview to 41%, a 13-point gain. Mistral also says output tokens on Terminal-Bench fell by 30% compared with the preview. Shattered notes that CNBC’s October 6 report had described the preview as the strongest open-weight model built outside China while saying it lagged the frontier in areas such as coding; three weeks later, Shattered writes, “the coding story reads differently”. At preview, Artificial Analysis had scored the model 38 overall, as covered in Unicorn’s first article on Large 4.

The release also ships three reasoning modes, low, medium and high, letting users trade effort against task complexity. Shattered says high is the default.

Benchmark	Mistral Large 4 result	Basis
Artificial Analysis Coding Agent Index	48 (level with GPT-6.1 Terra and Codex)	Third-party index, as reported
Terminal-Bench 4.0	41% (28% in preview)	Mistral and coverage
FrontierCode 2.0	19.7%	Mistral, via The Decoder
Artificial Analysis Cyber Index	51%	Mistral, via The Decoder
BrowseComp (web browsing)	84.5%	Mistral
FinWorkBench	73.3%	Mistral

Shattered reads the low FrontierCode 2.0 figure as a sign of a deliberately hard test that “leaves headroom for every vendor”, not of a weak model. Mistral also cites Surge Code human-preference results in which testers favoured Large 4 over GLM-5.3, Kimi K3 and DeepSeek V4.1 Flash; Gigazine reports a win rate above 50% against Chinese-made models in that evaluation. Those preference results are Mistral’s own citation, not an index score.

The token-efficiency claim behind the ranking

Mistral pairs the coding score with a cost argument. In the company’s words, Large 4 “rivals top open models such as GLM-5.3 on the Artificial Analysis Coding Agent Index while using 47% less tokens per task on average”. Shattered says SiliconANGLE names GLM-5.3 as the competitor in that comparison, while Gigazine phrases it more loosely as fewer tokens than rivals, so the narrow reading against GLM-5.3 is the safest one.

For agent workloads, token counts matter twice: fewer output tokens lower the bill and shorten completion times when an agent loops through dozens of steps. ZDNET makes the broader point that as proprietary AI “remains expensive to use, economical models that don’t sacrifice performance are eye-catching”. The 47% figure is Mistral’s claim; the sources do not show an independent check of it.

Specs, access and licence

Large 4 is a mixture-of-experts model, an architecture in which only some sub-networks run for each step, with roughly one trillion total parameters and, according to The Decoder and SiliconANGLE, 52 billion active per step. Other published specs include a context window of up to one million tokens, native multimodal input, fluency in more than 160 languages, and text-and-image input with text-and-code output. Mistral says the model “was trained from scratch on 4,000 Nvidia Grace Blackwell GPUs over two months, deployed in Mistral’s own data centers in Europe”.

On access, Shattered reports that the weights are on Hugging Face and the model is also available through Mistral Studio, the Mistral API, OpenRouter, NVIDIA endpoints and Microsoft Foundry. The licence lets most companies run the model on their own hardware but, per Shattered, “carries a catch for cloud resellers”; the sources here do not spell out the terms. A technical report is listed as coming soon, so several figures still rest on Mistral’s launch materials and on third-party indexes relaying them.

What hands-on testing found, and what it means for builders

Independent hands-on evidence so far predates the extra training. MindStudio tested the preview on October 8 through Open Code, an agentic coding interface, via OpenRouter, using eight fixed tasks called KingBench 3. The model scored 42 out of 80. Strong results included a three-elevator simulation (9.5/10) and a local Gemma 2B fine-tune with a web page, scored 10/10, in which validation loss fell from about 7.97 to 0.31 over 400 LoRA training steps. LoRA is a lightweight method for fine-tuning a model by training small add-on weights.

The weakness was reliability. On two tasks the model ended its turn without delivering a file or an answer and scored zero. One of those, a 3D contact lens case, worked on a rerun with reasoning disabled in Open Code, which MindStudio attributes partly to a tooling artifact. A 3D wristwatch build became usable only after the tester fed the browser error back to the model. Whether the open-weights build fixes those stalls is not shown by any source here.

For teams choosing a coding model, the practical points from the sources are:

• Composite parity, not across-the-board lead. The model is level with GPT-6.1 Terra on one agentic index and roughly level with GLM-5.3, but trails top Chinese open models on overall intelligence.
• Self-hosting is the differentiator. Mistral pitches the model as security “that stays under user control”, unlike proprietary models whose access labs can withdraw.
• Test the reasoning mode. The preview’s failures were sensitive to reasoning settings, and the three modes trade accuracy against spend per task.
• Check the licence before reselling. The reported cloud-reseller restriction matters for platforms that host models for others.

What to watch next

Three things will firm up or revise this picture. The first is Mistral’s technical report, which should put its own benchmark figures, including the 47% token saving, on a documented footing. The second is the Artificial Analysis leaderboard itself, which would settle the gap between ZDNET’s two statements and confirm whether Large 4 sits at 48 alongside GPT-6.1 Terra or a point behind GLM-5.3 among open models. The third is fresh hands-on testing of the open-weights build, to see whether the stalled turns MindStudio found in the preview persist after three more weeks of reinforcement learning.