ASR (speech recognition)

ASR, or automatic speech recognition, is the system that converts live spoken audio into text so an avatar or agent can understand the user and decide what to do next.

How ASR works in real-time avatars

ASR stands for Automatic Speech Recognition. It converts the user's spoken audio into text that the model or agent can understand. In a live avatar session, ASR is the first stage of the conversation loop.

Streaming ASR is especially important for real-time systems because it can produce partial transcripts while the user is still speaking. That lets the agent prepare a response earlier, instead of waiting for a full recording to finish.

A concrete example: a user says, "Can I connect this to Salesforce?" The ASR system turns that audio into text, the agent checks the integration answer, and the avatar responds through voice and face animation.

ASR quality affects both accuracy and speed. Background noise, accents, product names and technical vocabulary can all change the result, so production avatar systems need speech recognition that fits the environment where the agent will be used.

What Anam ships

Anam's Cara-4 model delivers expressive real-time avatars with around 150 ms server-side avatar-generation latency once a session is running, across 70+ languages. Builders use JavaScript and Python SDKs or integrations for LiveKit, Pipecat, ElevenLabs Agents, Agora, and VideoSDK. Bring any AI stack including OpenAI, Claude, Gemini, Mistral, Groq, Deepgram, Cartesia, or custom providers. The platform supports WebRTC delivery, SOC 2 Type II, HIPAA, zero data retention, and regional data residency. Sessions stream low-latency audio and video to browsers and native apps.

Explore the real-time avatar API →

Related terms

AI voice agent

An AI voice agent is a speech-first conversational agent that uses speech recognition, an LLM or agent workflow, and text-to-speech to understand spoken input and reply naturally in real time.

Turn detection

Turn detection is the process of deciding when a user has finished speaking, so the avatar can respond at the right moment without interrupting or waiting too long.

Barge-in

Barge-in is the ability for a user to interrupt an avatar while it is speaking, so the system can stop the current response, listen to the new input, and continue the conversation naturally.

Sub-second latency

Sub-second latency is the end-to-end response time under one second between a user speaking and an avatar replying, covering speech recognition, model reasoning, speech synthesis, animation, and network streaming.

Frequently asked questions

What does ASR do in a real-time avatar session?

ASR turns the user's live speech into text, which the agent or language model can then interpret before deciding what the avatar should say next.

Why does streaming ASR matter?

Streaming ASR can produce partial transcripts while the user is still speaking. That helps the agent prepare earlier and reduces the total response delay.

What makes ASR difficult for avatar agents?

Noise, accents, interruptions, product names, and domain-specific vocabulary can all reduce accuracy. Errors at this stage can send the entire conversation in the wrong direction.

Is ASR the same as voice activity detection?

No. ASR converts speech to text. Voice activity detection identifies when someone is speaking, which helps with turn-taking and barge-in.

Last updated: 17th July 2026 · Reviewed quarterly.

← Back to glossary