Lip sync

Lip sync is the process of aligning an avatar's mouth shapes and facial motion with generated speech audio, so the character appears to speak the words at the right time.

How lip sync works

Lip sync is the process of matching the avatar's mouth movements to the speech audio. The system needs to know which sounds are being spoken and animate the face so the mouth shapes land at the right time.

In a real-time avatar, lip sync has to happen while the response is being generated and streamed. The avatar cannot wait for a finished audio file if the goal is a live conversation.

A concrete example: when the avatar says "WebRTC", the mouth needs to form the right sequence of shapes as the word is spoken, not a fraction of a second early or late.

Good lip sync is mostly invisible. Users notice it when it breaks: the face feels dubbed, delayed or detached from the voice. That is why lip sync sits so close to latency and rendering quality in avatar systems.

What Anam ships

Anam's Cara-4 model delivers expressive real-time avatars with around 150 ms server-side avatar-generation latency once a session is running, across 70+ languages. Builders use JavaScript and Python SDKs or integrations for LiveKit, Pipecat, ElevenLabs Agents, Agora, and VideoSDK. Bring any AI stack including OpenAI, Claude, Gemini, Mistral, Groq, Deepgram, Cartesia, or custom providers. The platform supports WebRTC delivery, SOC 2 Type II, HIPAA, zero data retention, and regional data residency. Sessions stream low-latency audio and video to browsers and native apps.

Explore the real-time avatar API →

Related terms

TTS

TTS, or text-to-speech, is the system that converts a model's text response into spoken audio, giving an avatar a voice that can be streamed and lip-synced in real time.

ASR (speech recognition)

ASR, or automatic speech recognition, is the system that converts live spoken audio into text so an avatar or agent can understand the user and decide what to do next.

Real-time avatar

A real-time avatar is a photorealistic digital character that listens, thinks, and speaks back inside a live audio and video stream, typically with sub-second response latency, so it can hold a two-way conversation rather than play a pre-rendered clip.

Diffusion model

A diffusion model is a generative AI model that learns to create realistic images or video frames from noise, and can be used in avatar systems to help produce photorealistic visual output.

Frequently asked questions

What is lip sync in a real-time avatar?

Lip sync is the process of matching an avatar's mouth shapes to the speech audio, so words look like they are being spoken by the character in the video stream.

Why does lip sync matter for realism?

People notice mouth timing quickly. Even a photorealistic avatar feels artificial if the lips trail the voice, form the wrong shapes, or drift out of sync during a live turn.

Is lip sync only about the mouth?

No. Strong lip sync also depends on jaw motion, facial expression, head movement, blinking, and timing with the voice so the whole performance feels coherent.

What makes real-time lip sync difficult?

The system has to animate speech as audio is generated, often before the full sentence is known. It needs accuracy, stability, and low latency at the same time.

Last updated: 17th July 2026 · Reviewed quarterly.

← Back to glossary