Qwen3.8 Omni Flash Realtime API

Qwen3.8 Omni Flash Realtime is the live voice member of Alibaba's Qwen3.8 family: a model you talk to rather than type at. You stream microphone audio, and optionally camera frames, into an open connection, and it answers out loud while the conversation is still going. It listens while it speaks, stops when you cut in, and calls tools in the middle of a sentence. The Qwen team describes it as built for continuous, low-latency interaction with smart devices, robots and interactive agents, and the release is aimed squarely at developers who want a voice assistant that behaves like a conversation partner and not like a walkie-talkie.

The headline numbers are about speed and listening. The Qwen technical report measures roughly one second from the end of your speech to the first audio of the reply, and about 1.2 to 1.35 seconds when video frames are part of the input. The model recognises speech in 74 languages and 39 Chinese dialects, and speaks 29 languages and 7 dialects. It also does something few voice models attempt: given multichannel audio, it works out which direction a sound is coming from and how far away it is.

What makes the launch interesting is the jump over the previous Qwen omni model. On OmniVChat-Bench, a test of spoken multimodal conversation, the realtime model scores 82.3 against 52.1 for Qwen3.5-Omni-Plus. That is the gap between a demo and something you can put in front of customers.

Where it comes from: the Qwen3.8 Omni release

Qwen is Alibaba's large-model team, and its omni line is the part of the family that takes in audio, images and video natively instead of bolting a speech recogniser and a text-to-speech engine onto a text model. The Qwen3.8 Omni release in September 2026 ships as two models that share a name and a backbone but do different jobs.

Qwen3.8 Omni Flash is the offline variant. It reads complete audio and video files, reasons over them with a context of up to a million tokens, and answers in text. It is the one for transcribing a meeting, summarising a lecture recording or editing a video.

Qwen3.8 Omni Flash Realtime, the model on this page, is the streaming variant. It holds a live, two-way session, takes audio and frames as they arrive, and replies with speech plus a text transcript. Early coverage pointed out that the shared "Omni Flash" name is easy to confuse: the non-realtime model does not produce audio at all, so anyone building a voice product wants the Realtime id.

Both are closed models. Alibaba has not published weights or a parameter count, and access is through its cloud API and through API providers such as Unicorn. Alongside the models the team did open-source two pieces of tooling: Qwen-Live-Harness, a framework for building real-time multimodal agents with asynchronous tool use, proactive behaviour and persistent memory, and Qwen-MM-Plugins, a set of plugins that fill gaps in audio and video support for agent frameworks.

What changed from Qwen3.5-Omni-Plus

The technical report names Qwen3.5-Omni-Plus as the model this release replaces, and the differences go beyond a benchmark bump.

• Full-duplex by design. The model processes input in chunks while it is generating, so it keeps hearing you while it talks. When you start speaking, the reply can be cancelled on the spot. An interruption stops the speech without cancelling tasks the model has already handed off, so a tool call that is fetching an order status keeps running while you change the subject.
• Better conversation scores. On OmniVChat-Bench it reaches 82.3, up from 52.1, and ahead of Gemini 3.8 Flash at 65.6 in Qwen's own table. On the audio track of ODUbench it scores 90.2, against 74.7 for its predecessor and 72.1 for Gemini 3.8 Flash.
• Spatial hearing. With two- or four-channel audio, the model receives first-order ambisonic signals that preserve the timing and level differences between microphones. Qwen says this makes it the first omni model that can locate a target by sound, combining the direction and distance of a voice with what the camera shows about obstacles and open space.
• Pronunciation awareness. The model jointly models how a word sounds and what it means. It understands non-standard or heavily accented speech, maps it to the intended words, and can produce a standard-pronunciation version on the spot, which turns it into a speaking coach as well as a listener.
• Smarter turn detection. Alongside plain server-side voice activity detection, a semantic mode filters out backchannel noises like "mm-hm" and background chatter, so the model is less likely to think you have finished, or started, speaking when you have not.
• Tools in the call. Function calling, remote MCP tools and web search are all available from inside a realtime session.
• Custom voices. Besides the preset voices (the default is Tina), the platform supports cloned voices. The report puts voice-cloning word error rates at under one per cent on its main test sets.

The family-level picture helps too. On the offline Omni Flash variant, Qwen reports speaker diarization error on the AliMeeting meeting corpus falling to 3.4, against 72.6 for Gemini 3.8 Flash (lower is better). That is a file-based result, not a realtime one, but it shows how much work went into the shared audio front end.

How it works: a Thinker and a Talker

Qwen's omni models split the job in two. A Thinker handles understanding and reasoning; in Qwen3.8 it is a sparse mixture-of-experts network inherited from the Qwen3.8-Next text model, co-trained on text, audio, images and video so it keeps its text skills. A Talker turns the Thinker's output into speech.

The Talker predicts residual vector-quantised audio tokens and uses multi-token prediction to fill in the extra codebooks quickly. A causal Code2Wav decoder then reconstructs the waveform incrementally at 48 kHz, and an alignment method the report calls ARIA lets speech continue from a partial text prefix. In the report's words, text generation, speech-token prediction and waveform reconstruction proceed progressively, so audio starts playing before the full reply exists. The measured result is text at about 81 to 85 tokens per second and speech generated around 6.5 times faster than real time.

Specs at a glance

Where it shines

Conversations that feel live. The combination of about one second to first audio, cancellation the moment you speak and semantic turn detection is what separates a usable voice agent from a frustrating one. A customer who says "no, the other order" halfway through a reply gets an immediate correction instead of a finished monologue.

Wide, dialect-aware listening. Recognition across 74 languages and 39 Chinese dialects, including Cantonese, Sichuanese, Shanghainese and Southern Min, makes it a strong fit for services in China and Southeast Asia, and for any app whose users do not all speak textbook English or Mandarin.

Agents that do things while talking. Because tool calls and speech run side by side, the model can say "let me check that" and keep the conversation going while a function runs. Food ordering, bookings, account lookups and smart-home control are the natural use cases.

Seeing as well as hearing. Frames from a phone or device camera become part of the context, so a user can point at a dish, a broken appliance or a page of homework and ask about it.

Language practice. The pronunciation modelling means it can hear a learner's attempt, understand what they meant and say it back correctly, which suits tutoring apps.

Limits worth knowing

VoiceBench went down. On VoiceBench, a broad test of spoken-assistant ability, the realtime model scores 88.8, below both Qwen3.5-Omni-Plus at 92.9 and Gemini 3.8 Flash at 92.3. The gains are in multimodal conversation, not across the board.

It hears more languages than it speaks. Recognition covers 113 language and dialect varieties, speech output only 36. A user can talk to it in a language it understands but cannot answer in aloud; plan for text fallback in those cases.

Short media memory. The context holds nearly 200,000 tokens, but the audio history is capped at 600 seconds and the video history at 240 seconds. In a long call the model keeps the text of what was said but loses the oldest raw audio and frames.

Frames, not motion video. Video arrives as roughly one image per second. That is fine for "what is this?" and poor for anything that depends on fast movement.

Early stability reports. Some early testers reported frequent interruptions and unstable sessions when wiring it into coding-agent tools, and independent benchmarks were still scarce at launch. Test it with your own audio conditions before relying on it.

Closed and opaque. No weights, no parameter count, and no self-hosting.

Building with it

The model is driven by a session, not a single prompt. You open a WebSocket, wait for session.created, send a session.update with your settings, then stream audio with input_audio_buffer.append. Replies arrive as response.audio.delta chunks for speech and response.audio_transcript.delta for the text. A few habits help:

• Write instructions for the ear. Ask for short sentences, no lists or Markdown, and numbers said the way a person would say them.
• Use server turn detection and let the model handle interruptions rather than building your own push-to-talk.
• Keep tool results compact. The model reads them aloud in its own words, so give it facts, not formatted tables.
• Turn on input transcription if you want a written record of what the user said.

Example instructions for a session:

You are the voice assistant of a neighbourhood bakery. Speak in short, friendly sentences and never read out lists. Confirm each order item back to the customer before you call place_order. If the customer interrupts, stop and listen.

You are a patient Mandarin speaking coach. When the learner says a sentence, tell them in English what you understood, then say the sentence again slowly with standard pronunciation and ask them to repeat it.

You can see through the user's phone camera. Describe only what is needed to answer their question, and say so plainly if the picture is too dark or blurred to tell.

How it compares with other models on Unicorn

Most chat models in the catalog are text models reached through chat completions. Gemini 3.8 Flash is the closest rival in Qwen's own tables: on Unicorn it reads text, images and video and writes text, and it leads the realtime Qwen on VoiceBench while trailing it on OmniVChat-Bench and ODUbench. Pick Gemini 3.8 Flash for agentic coding and long-document work, and Qwen3.8 Omni Flash Realtime when the user is speaking and expects to hear an answer.

For deep reasoning behind a voice front end, a common pattern is to let the realtime model handle the conversation and call a text model as a tool. Claude Opus 5 and GPT-6 Astra are the heavyweight choices for that back end; Claude Haiku 4.5 suits quick lookups where speed matters more than depth.

Getting started on Unicorn API

On Unicorn API the model id is qwen3-8-omni-flash-realtime, served on the /v1/realtime WebSocket endpoint rather than on chat completions. Unicorn relays the model's realtime protocol unchanged, so the session events, server turn detection, transcripts and tool calls described above work as documented. Audio goes in as 16 kHz, 16-bit mono PCM and comes back as 24 kHz, 16-bit mono PCM, both base64-encoded. A call can last up to 60 minutes, closes after 5 minutes without any client message, and each person can have 3 calls open at once.

The quickest way to try it is the Voice playground: sign in, allow the microphone and start talking, with a live transcript of both sides. To build your own app, create a key on the API keys page, keep it on your server, and open the connection from there with the key in the Authorization header. Browsers cannot send that header on a WebSocket, so a signed-in page first asks /v1/realtime/sessions for a one-use ticket and connects with that. The API docs cover the endpoint, the event that reports each reply's usage, and the error and close codes.

Verdict

Qwen3.8 Omni Flash Realtime is the most capable live voice model the Qwen team has shipped. Replies start in about a second, interruptions work the way people expect, the language and dialect coverage for listening is wide, and tool calls and camera frames sit naturally inside a conversation. It is closed, it speaks fewer languages than it understands, its media memory is short, and it slips on VoiceBench. For voice assistants, tutors and device agents, especially ones serving Chinese and Asian-language users, it is an easy model to shortlist.