VAD (voice activity detection)

VAD, or voice activity detection, is the process of detecting when a user is speaking or silent, helping an avatar manage turn-taking, barge-in, and response timing during a live session.

How VAD works

VAD stands for voice activity detection. It detects whether a live audio stream contains speech or silence, usually without trying to transcribe the words.

In real-time avatars, VAD helps the system decide when the user has started speaking, paused, interrupted, or gone quiet. It often works alongside ASR and turn-taking logic.

A concrete example: while an avatar is speaking, VAD notices that the user has started talking and signals the system to stop the current response so barge-in can happen.

Good VAD is sensitive enough to catch real speech but not so sensitive that keyboard clicks, background noise, or breathing constantly interrupt the avatar.

Diagram explaining how the Anam Engine is used for AI developers

Diagram of how Anam works

What Anam ships

Anam's Cara-4 model delivers expressive real-time avatars with around 150 ms server-side avatar-generation latency once a session is running, across 70+ languages. Builders use JavaScript and Python SDKs or integrations for LiveKit, Pipecat, ElevenLabs Agents, Agora, and VideoSDK. Bring any AI stack including OpenAI, Claude, Gemini, Mistral, Groq, Deepgram, Cartesia, or custom providers. The platform supports WebRTC delivery, SOC 2 Type II, HIPAA, zero data retention, and regional data residency. Sessions stream low-latency audio and video to browsers and native apps.

Explore the real-time avatar API →

Related terms

Turn-taking

Turn-taking is the timing system that decides when the user is speaking, when the avatar should listen, and when the avatar should respond, so the conversation feels natural rather than rushed or delayed.

Barge-in

Barge-in is the ability for a user to interrupt an avatar while it is speaking, so the system can stop the current response, listen to the new input, and continue the conversation naturally.

ASR (speech recognition)

ASR, or automatic speech recognition, is the system that converts live spoken audio into text so an avatar or agent can understand the user and decide what to do next.

Sub-second latency

Sub-second latency is the end-to-end response time under one second between a user speaking and an avatar replying, covering speech recognition, model reasoning, speech synthesis, animation, and network streaming.

Frequently asked questions

What does VAD do in an avatar session?

VAD detects when the user is speaking or silent, helping the avatar know when to listen, wait, interrupt itself, or prepare a response.

Is VAD the same as speech recognition?

No. VAD detects speech activity. Speech recognition converts spoken words into text. Real-time avatars often need both.

Why does VAD matter for barge-in?

Barge-in depends on knowing when the user starts speaking while the avatar is still talking. VAD provides the signal that an interruption may be happening.

What makes VAD difficult in production?

Background noise, quiet speakers, overlapping audio, accents, microphones, and network conditions can all make it harder to distinguish speech from non-speech reliably.

Last updated: 17th July 2026 · Reviewed quarterly.

← Back to glossary

Try the real-time avatar API trusted by 8,000 builders

Read the docs

HIPAA & SOC 2 Certified