Microsoft open-sources Mage-VL-Cowork, a 4B agent that operates a PC

Microsoft has published Mage-VL-Cowork, a 4-billion-parameter vision-language model trained to operate a computer, on Hugging Face under the Apache-2.0 licence. Microsoft’s model card reports that it completes 66.0% of tasks on OSWorld-Verified, a standard test of desktop agents, which it describes as the best result among 4B models. That figure was measured by the team that built the model, and nobody else has repeated it yet.

The interesting part is not only the size. Mage-VL-Cowork decides at every step whether to click through the screen or to drop into a terminal and write a script, without a separate program routing between the two.

What Microsoft released on Hugging Face

According to coverage, Microsoft put the model on Hugging Face on October 30, 2026. A computer-use agent receives an instruction, looks at a screenshot of the desktop and decides the next move: click, type, open a menu or run a command. Mage-VL-Cowork takes a screenshot plus a task written in English, reasons about it and answers with one precise move.

Under the hood sits a vision encoder called Mage-ViT, which reads the images, connected to Qwen3-4B-Instruct-2507, Alibaba’s text model, which handles language. The starting point is Mage-VL, a multimodal model that Microsoft’s Mage team uploaded on July 25, 2026. The model card describes the release as “a full training recipe to learn hybrid computer use from evolved harnesses”.

Spec	Mage-VL-Cowork
Parameters	4 billion
Licence	Apache-2.0
Backbone	Mage-ViT vision encoder + Qwen3-4B-Instruct-2507
Context window	32,768 tokens active; architecture reaches 262,144
Weights	Two bfloat16 files, 9.48 GB total
Action channels	computer_use (11 interface actions) and bash
OSWorld-Verified	66.0%, self-reported

Not the same thing as Copilot Cowork

The name invites confusion. Microsoft already sells Copilot Cowork, an agent inside Microsoft 365 that turns a request into a plan and carries it out across Outlook, Teams, Excel and other apps, with checkpoints for the user to approve changes. Microsoft says it built that product working closely with Anthropic, integrating the technology behind Claude Cowork into Microsoft 365 Copilot. The Mage-VL-Cowork model card does not mention Copilot Cowork and does not say the model is part of it; coverage describes it as a research model from the Mage team, not a Microsoft 365 feature.

How Mage-VL-Cowork chooses between clicking and coding

The model has two channels for action. The first, computer_use, offers 11 interface actions such as click, double-click, drag, typing, scrolling and pause, with screen coordinates normalised to a 0-999 range. The second, bash, opens a persistent shell with Python and returns only text, so it does not cost an extra screenshot.

Each turn, the model emits three parts: a <think> rationale, a one-line Action: summary, and a <tool_call> that is either a GUI action or a shell command. The chat template in the Hugging Face repository asks for each tool call as a JSON object with a function name and arguments inside <tool_call> tags, and includes a switch that turns the thinking block off. The choice of channel is learned, not handled by an external router or a second model.

The split is practical. Pixel-only agents spend thousands of tokens on every screenshot, and a spreadsheet edit can turn into dozens of mouse clicks. When the job is visual, such as touching up a layer in GIMP or positioning a chart, the model clicks; when it is structured, such as spreadsheet cells, slides or documents, it writes a script. The model card’s example is an Excel file: the model notices LibreOffice Calc is not even open, reasons that navigating menus would be slower, and merges the cells with openpyxl, the Python library for xlsx files.

To keep long tasks affordable, the card describes an asymmetric working memory: the three most recent screenshots and four full responses stay in context, while older steps are folded into one-line action summaries.

What the 66% on OSWorld-Verified means

OSWorld is the most widely used test bed for these agents because it asks them to finish real tasks inside a real operating system, across spreadsheets, image editors, browsers and files, and checks the result with a script. Microsoft used the Verified version with 361 tasks, leaving out those that require Google Drive.

The 66.0% is the average of two full passes that scored 65.33% and 66.75%, run within 50 steps at 1920×1080 resolution with Microsoft’s own agent setup. The human reference in the same table is 72.4%. In that table, larger models whose size is not disclosed still score higher:

Model (as listed in Microsoft’s table)	OSWorld-Verified
GPT-5.5	78.7%
Claude Opus 4.7	78.0%
GPT-5.4	75.0%
Human reference	72.4%
GPT-5.4-mini (single pass)	69.6%
Mage-VL-Cowork (4B)	66.0%

The claim, then, is about efficiency at small size, not about beating frontier models. Coverage is explicit that the benchmark is not independent: the numbers are Microsoft’s own, from the model card.

The side-by-side demos

Microsoft also filmed two runs against Qwen3.6-27B, a model almost seven times larger that works through the graphical interface alone. On a Calc task (merge two ranges, add a column chart, save as xlsx), Mage-VL-Cowork finished in 39 seconds and 6 steps, against 9 minutes 56 seconds and 39 steps for the rival. On a GIMP task, it took 1 minute 58 seconds, while Qwen3.6-27B stopped at 7 minutes with a file that failed verification. Both tasks were picked by Microsoft’s team, so they count as demonstrations rather than evidence.

How Microsoft trained a small agent to do this

The training recipe has three parts, and coverage calls the second, the Mage-CUA-Forge task synthesis, arguably the most reusable.

1. Cleaning up the teacher’s tools

Before collecting training data, Microsoft used an authoring agent to refine the scaffold, or harness, around the teacher model: its tool interfaces, system prompt and pre-installed terminal environment. It removed overlapping tools such as xdotool and verbose accessibility-tree dumps, and pre-installed Python libraries including python-pptx, openpyxl, pandas, pypdf and pillow.

In a controlled comparison with a GPT-5.4 teacher on 672 held-out multi-application tasks, the pruned harness gave the teacher 92.0% success in 25.0 mean steps, against 89.6% in 35.33 steps with accessibility data added. The larger effect showed up in the student: a 4B model trained on the cleaner trajectories reached 37.6%, against 25.1% for one trained on the cluttered harness. In Microsoft’s words:

more tools do not automatically make a better agent

Coverage adds that this applies especially to small students.

2. Mage-CUA-Forge: generating tasks from reusable workflows

Instead of running an open-ended agent loop to invent every training task, Mage-CUA-Forge writes a reusable generation workflow once per atomic skill. Per the card, three rounds of student feedback produced 293,806 tasks across 6,915 atomic skills and 278 file categories, and kept 318,837 verified hybrid trajectories, 68.19% GUI tool calls and 31.81% terminal calls. Microsoft says that with the same generator model this cut tokens per task from 1,276K to 85K and recurring spend from $0.906 to $0.158 per task. That is a claim about Microsoft’s own pipeline.

3. Specialise, then unify

The model first learns GUI skills and coding-terminal skills separately, then learns to choose between them, across five stages ending in online reinforcement learning with verifiable rewards on 1,988 tasks. Total nominal compute was 7.79 ZFLOPs on 32 NVIDIA B200 GPUs plus 1,120 CPU sandbox slots for the reinforcement-learning stage. OSWorld-Verified scores rose from 45.56% after GUI fine-tuning to 55.84% after hybrid fine-tuning and 66.04% after reinforcement learning, which also cut model calls per task from 21.57 to 18.77.

How to run it, and the risks

The weights are free to download from Hugging Face, and coverage points to evaluation code on GitHub. It can be served with an SGLang Docker image or loaded through Transformers with trust_remote_code=True. That flag lets the repository’s own code run on the host, which is one more reason to review what is loaded.

A GPU is needed, since the weights take up 9.48 GB, and so is an isolated environment such as a Docker container or a virtual machine, because the agent runs shell commands. An agent with a persistent shell can delete files or change system settings as easily as it can edit a spreadsheet. Coverage suggests the model’s best uses today are research, local agent prototypes and low-cost spreadsheet and document automation inside a sandbox.

What it means for builders

• Local, self-hosted agents become more realistic. A 4B model at under 10 GB of weights can run on a single GPU, under a licence that permits commercial use.
• Scripts beat clicks for structured files. The design argues that agents should edit spreadsheets and documents through code when they can, and keep the screen for visual work.
• Harness design matters as much as the model. Microsoft’s own comparison suggests fewer, cleaner tools produce better training data for small models.
• Treat the numbers as claims. The benchmark score, the cost reductions and the demos all come from Microsoft, and the demo tasks were chosen by the team.

Open questions remain: how the model performs outside OSWorld, whether the hybrid approach holds up on software that cannot be scripted, and how it behaves with its 32,768-token active context on long, messy tasks.

What to watch next

The first thing to watch is whether independent testers reproduce the 66.0% OSWorld-Verified score, since no one has done so yet. The second is whether other teams adopt the Mage-CUA-Forge approach of writing one task-generation workflow per skill, which coverage singles out as arguably the most reusable part of the work. Finally, the gap to the 72.4% human reference and to larger models in the high 70s is the line small open agents will now be measured against.