Sarvam AI open-sources Prism, a model that picks answers in 44 ms

Sarvam AI, the Bengaluru startup, has released Prism, an open-weight “decision model” that returns a probability for each of a set of answer options instead of writing a reply, and does so in about 44 milliseconds. Prism is a LoRA fine-tune of Sarvam’s own 105-billion-parameter mixture-of-experts model, released under the Apache 2.0 licence on Hugging Face.

The pitch is speed for narrow choices: whether a transaction looks like fraud, which of several agents should take a task, or how confident a check should be. The details below come from Startup Fortune’s coverage, which in turn cites AlphaSignal’s reporting and Sarvam’s own figures; no independent evaluation of Prism is available yet.

What Sarvam Prism does differently from a chat model

A normal language model writes its answer token by token, scoring tens of thousands of candidate words at every step. Prism skips that. It computes scores directly from the model’s hidden states (the internal numerical representation the model builds of the input) and passes them to a small decision head that turns them into one probability per option. The options can be “fraud” versus “not fraud”, a choice among five possible agents, or a point on a confidence scale.

Because no sentence is generated, there is nothing to wait on, and the computation shrinks to the handful of tokens an answer actually needs rather than the model’s full vocabulary.

Two terms are worth defining. LoRA (low-rank adaptation) is a fine-tuning method that trains a small set of extra weights on top of a frozen base model rather than retraining the whole thing. A mixture-of-experts model routes each input through a subset of specialised sub-networks, so not all 105 billion parameters are active at once. The sources describe Prism’s base only as Sarvam’s 105B mixture-of-experts model and do not give the active parameter count.

Prism at a glance

Spec	Sarvam Prism (as reported)
Maker	Sarvam AI, Bengaluru
Base model	Sarvam’s 105B mixture-of-experts model
Training method	LoRA fine-tune
Output	One calibrated probability per answer option; no text
Reported latency	About 44 ms
Licence	Apache 2.0
Weight format	Custom FP8 layout
Where to get it	Hugging Face; Sarvam’s hosted Indus token factory
Languages tested for calibration	English and Indian languages

Where Prism fits among this autumn’s decision models

Prism arrives in a category that has filled up quickly. A decision model, as one developer’s October survey puts it, is “a language model that has been forbidden to write”: it takes a document and a typed question and returns a typed answer, such as a probability that a statement is true, a distribution over listed options, or a position on a scale. Recent research papers call these “System One” decision models, borrowing the psychology term for fast, intuitive judgement.

On October 6, OpenAI shipped a dedicated Decisions API running a single model, gpt-6-luna. It joins TypeSafe AI’s Jev, in early access since September 15, and two open-weight models: Convai Innovations’ Laya and AWS Strands Labs’ Strands Decider 2B.

Model	Weights	Size
Sarvam Prism	Open, Apache 2.0	LoRA on a 105B MoE base
OpenAI Decisions API (gpt-6-luna)	Closed, API only	Not published
Jev (TypeSafe AI)	Closed, API only	Not published
Laya (Convai Innovations)	Open, Apache 2.0	421M, ModernBERT-large base
Strands Decider 2B (AWS)	Open, Apache 2.0	About 2B, Qwen3.5-2B base

What stands out is scale. The other open-weight options are small: Laya has 421 million parameters, and Strands Decider is a Qwen3.5-2B whose text-generating head was replaced with a pointer head of about a million parameters that scores supplied options. Prism instead puts a decision head on a 105B-class model. Whether that extra capacity buys better judgement on real tasks is not something the sources test head to head.

The approach itself is also becoming easier to reproduce. A paper by Yinheng Li and Justin Wagle, published October 4, argues that any LLM can be turned into a Jev-style decision model, and offers a guide on when off-the-shelf LLMs are enough and when and how to fine-tune them.

Sarvam’s speed and accuracy claims, and what backs them

On accuracy, Prism scored an average of 71.8 across three decision-accuracy indices, which Sarvam’s own reporting frames as the best result among models answering in under 100 milliseconds, according to AlphaSignal as cited by Startup Fortune. The sources do not name the three indices or list the competing models, so the comparison cannot be checked from them.

On speed, Startup Fortune sets Prism’s 44 ms against Groq’s and Cerebras’s custom inference hardware, which it says brings language model latency down to roughly 80 to 150 milliseconds time-to-first-token on a narrow catalogue of ported models. That is the outlet’s comparison, and it is not like for like: time-to-first-token measures when a generating model starts writing, while Prism never writes at all. The outlet itself attributes Prism’s speed to skipping text generation rather than to hardware. The sources do not say what hardware, batch size or input length the 44 ms figure was measured on.

Sarvam also disclosed a less flattering result: Prism is underconfident across every confidence band it was tested on, in both English and Indian languages, meaning its actual error rates came in lower than its stated probabilities implied. Calibration is the property that a model’s “0.8” really means it is right about eight times in ten. Startup Fortune notes that underconfidence is a safer failure mode for a fraud filter than overconfidence.

How to run Prism, and the integration catch

Prism is available two ways: as downloadable weights on Hugging Face and through Sarvam’s hosted Indus token factory. The Apache 2.0 licence allows commercial use and modification.

Self-hosting is not plug-and-play. Prism’s custom FP8 layout (FP8 is an 8-bit number format that cuts memory use and speeds up inference) is not compatible with standard tools such as Hugging Face’s AutoModelForCausalLM loader or general-purpose inference servers, according to AlphaSignal’s reporting. Teams either run it through Sarvam’s own serving stack or take on nontrivial integration work.

That is a change from Sarvam’s earlier open releases. Its Sarvam-1 v0.5 model, published on Hugging Face in 2024, is a roughly 2.5-billion-parameter text-generation model listed as loading through the standard Transformers AutoModelForCausalLM class.

What Prism means for builders working on agents

Sarvam is not aiming Prism at chatbots. Its target is the orchestration layer of agent systems: deciding which sub-agent handles a task, whether a transaction looks suspicious, or which response variant a support bot should send. The company says it already uses Prism internally for trace evaluation, prompt guardrails, routing decisions inside its token factory, and judging outputs in its speech-to-speech testing harness. In those spots, a delay of 500 milliseconds that would go unnoticed in a chat window becomes a bottleneck when it repeats hundreds of times in one automated workflow.

Practical points for teams weighing it:

• Output you can branch on. A single probability per option is easier to threshold in code than parsing a generated paragraph.
• Calibration is still your job. The October survey of four decision-model vendors found that each one’s documentation, read to the end, leaves checking whether a 0.8 really means eight in ten to the user. Sarvam’s underconfidence finding is a useful starting point, not a substitute for testing on your own data.
• Test routers by changing the question. The Strands model card warns: “With the state and options fixed, a changed question often gets the same answer.” The same check is worth running on any decision model, Prism included.
• Not a security boundary. TypeSafe documents that adversarial input written to steer the model can move Jev’s answer, and the survey’s author concludes such models can rank and filter but cannot be the boundary. Nothing in the sources suggests Prism is different, so prompt-guardrail use should be layered with other controls.
• Small classifiers remain competitive on some tasks. In Red Hat’s comparison cited in the survey, IBM’s 125M Granite Guardian classifier answered in a median 33 ms on content safety. A 44 ms figure from a 105B-based model is fast, but not automatically the fastest option for a single narrow task.

Sarvam’s wider bet and the open questions

Startup Fortune frames Prism as part of a sovereignty strategy. Sarvam has raised roughly $300 million at a valuation of about $1.5 billion, according to LetsDataScience, with later reports suggesting a new round could push that toward $3 billion. The company is building a stack that runs from its 105B foundation model to the Indus agent platform, which bundles coding, document and work agents.

Trust in Sarvam’s claims has also been tested recently. The same outlet reports that a technical teardown of Sarvam Code, the coding agent unveiled at the company’s Epoch 2026 conference, found it shares OpenAI Codex’s exact commands and config schema and an unscrubbed “x-openai-subagent” string, raising attribution questions. That concerns a different product, but it is a reason to want independent checks of Prism’s numbers.

Open questions the sources leave unanswered:

• Which three decision-accuracy indices produced the 71.8 average, and which sub-100 ms models it was compared with.
• What hardware and conditions the 44 ms latency was measured under.
• How much GPU memory a self-hosted 105B-based Prism needs compared with 2B-class rivals.
• Whether support for standard inference servers will follow the custom FP8 release.

What to watch next

The first thing to watch is independent testing. Red Hat’s benchmark already compares decision models, trained classifiers and LLM judges on content safety and prompt injection, and Prism is an obvious candidate for the next round. A third-party check of the 44 ms figure and of Sarvam’s calibration claim in Indian languages would settle much of the uncertainty.

Second, tooling: whether community or Sarvam-built loaders make the custom FP8 weights usable in common serving stacks, which will decide how many teams outside Sarvam’s own platform try it. Third, the category itself: with OpenAI, AWS, TypeSafe and Convai already shipping decision models, and research showing general LLMs can be adapted into them, the competition is likely to shift from raw speed to proof that the probabilities can be trusted.