AI voice agent

An AI voice agent is a speech-first conversational agent that uses speech recognition, an LLM or agent workflow, and text-to-speech to understand spoken input and reply naturally in real time.

How AI voice agents work

An AI voice agent is a spoken interface to an LLM or agent workflow. The user talks, speech recognition turns the audio into text, the model decides what to say next, and text-to-speech turns the answer back into voice.

In a real-time avatar stack, the voice agent is often the conversational core. The avatar layer adds a face, expressions and lip sync on top of the same turn-taking loop.

A concrete example: a support voice agent answers a delivery question, checks order status through a tool call, then speaks the result back to the customer. With an avatar, the same flow becomes a live video agent.

The quality of a voice agent depends heavily on latency, interruption handling and grounding. If it cannot hear accurately, respond quickly or stay inside approved knowledge, adding a face will not fix the experience.

What Anam ships

Anam's Cara-4 model delivers expressive real-time avatars with around 150 ms server-side avatar-generation latency once a session is running, across 70+ languages. Builders use JavaScript and Python SDKs or integrations for LiveKit, Pipecat, ElevenLabs Agents, Agora, and VideoSDK. Bring any AI stack including OpenAI, Claude, Gemini, Mistral, Groq, Deepgram, Cartesia, or custom providers. The platform supports WebRTC delivery, SOC 2 Type II, HIPAA, zero data retention, and regional data residency. Sessions stream low-latency audio and video to browsers and native apps.

Explore the real-time avatar API →

Related terms

ASR (speech recognition)

ASR, or automatic speech recognition, is the system that converts live spoken audio into text so an avatar or agent can understand the user and decide what to do next.

TTS

TTS, or text-to-speech, is the system that converts a model's text response into spoken audio, giving an avatar a voice that can be streamed and lip-synced in real time.

Turn detection

Turn detection is the process of deciding when a user has finished speaking, so the avatar can respond at the right moment without interrupting or waiting too long.

Interactive avatar

An interactive avatar is a digital character that listens to a user and responds in real time, using speech, language model reasoning, and live animation to support a two-way conversation instead of a fixed script.

Frequently asked questions

What is the difference between an AI voice agent and an avatar agent?

An AI voice agent is speech-first and may have no visual layer. An avatar agent adds a live face, lip sync, expressions, and video streaming on top of the same conversational loop.

What components does an AI voice agent need?

It needs speech recognition, an LLM or agent workflow, text-to-speech, turn detection, interruption handling, and tool access if it needs to fetch data or take action.

Can an AI voice agent use my existing LLM stack?

Yes. Many voice agents connect to an existing model or agent framework, then add speech input, voice output, latency controls, and live session handling around it.

When should a voice agent become a video avatar?

Use a video avatar when the interaction benefits from presence, guidance, expression, or trust, such as onboarding, support, sales demos, training, and user education.

Last updated: 17th July 2026 · Reviewed quarterly.

← Back to glossary