Latency budget

A latency budget is the total response-time allowance split across speech recognition, model reasoning, tools, text-to-speech, rendering, and streaming so a real-time avatar can stay conversational.

How latency budgets work

A latency budget is the time allowance for each stage of a real-time avatar conversation. The total budget might be under one second, but that total has to be shared between speech recognition, model reasoning, tool calls, speech synthesis, animation and streaming.

The budget makes tradeoffs visible. If a tool call takes 500ms, the rest of the system has less room. If TTS waits for a full paragraph before speaking, the avatar may sound polished but feel slow.

A concrete example: a team might target 150ms for ASR partials, 300ms for the first model tokens, 200ms for TTS startup and the remaining time for rendering and network delivery.

Latency budgets matter because users experience the total, not the individual services. A real-time avatar feels conversational only when the whole loop closes quickly enough.

Response time

User experience

What Anam ships

Anam's Cara-4 model delivers expressive real-time avatars with around 150 ms server-side avatar-generation latency once a session is running, across 70+ languages. Builders use JavaScript and Python SDKs or integrations for LiveKit, Pipecat, ElevenLabs Agents, Agora, and VideoSDK. Bring any AI stack including OpenAI, Claude, Gemini, Mistral, Groq, Deepgram, Cartesia, or custom providers. The platform supports WebRTC delivery, SOC 2 Type II, HIPAA, zero data retention, and regional data residency. Sessions stream low-latency audio and video to browsers and native apps.

Explore the real-time avatar API →

Related terms

Sub-second latency

Sub-second latency is the end-to-end response time under one second between a user speaking and an avatar replying, covering speech recognition, model reasoning, speech synthesis, animation, and network streaming.

WebRTC

WebRTC is the browser standard for low-latency live audio and video streaming, commonly used to carry microphone input and avatar output during real-time interactive sessions.

SDP

SDP, or Session Description Protocol, is the WebRTC setup format that describes media tracks, codecs, and connection details so a browser and avatar service can start a live stream.

TTS

TTS, or text-to-speech, is the system that converts a model's text response into spoken audio, giving an avatar a voice that can be streamed and lip-synced in real time.

Frequently asked questions

What is included in a latency budget for avatars?

The full budget includes microphone capture, speech recognition, model reasoning, tool calls, text-to-speech, face animation, encoding, network delivery, and playback in the browser.

Why split avatar latency into a budget?

A single total latency number hides the bottleneck. A latency budget shows how much time each stage can spend before the whole conversation starts to feel slow.

Which stage usually causes the biggest delay?

It depends on the stack, but model reasoning, tool calls, TTS startup, and network jitter are common sources. The only reliable answer comes from measuring every stage separately.

What is a good latency budget for a live avatar?

For a natural conversation, aim for the whole loop to feel closer to 250ms than 800ms. That usually means streaming early and avoiding blocking work wherever possible.

Last updated: 17th July 2026 · Reviewed quarterly.

← Back to glossary