We cut unwanted avatar interruptions by around 50% by moving turn detection to Deepgram Flux, with no increase in latency. Here's what changed and why it matters more when there's a face involved.

People don't talk in neat turns. We stop to think, trail off, restart a sentence halfway through, and do all of it with a TV on in the next room. The person we're talking to copes without noticing.

An interactive avatar has to make that call many times in every conversation: has this person finished, or are they still working out what to say? Answer too early and the avatar talks over them. Answer too late and every reply starts with an awkward pause.

We've spent the last few months on this part of the Anam stack. In our testing, the changes cut unwanted avatar interruptions by around 50% with no added latency. Most of that came from moving turn detection to Deepgram Flux.

Why turn-taking is harder with a face

Turn-taking is something people do on autopilot. A pause can mean "I'm done". It can also mean "hang on, what's the word".

Standard speech-to-text tells you what someone said. It doesn't tell you whether they've finished saying it. The simple fix is a silence timer: wait a few hundred milliseconds, then assume the turn is over. Make it short and you cut people off. Make it long and the whole conversation feels sluggish.

A face makes the mistakes more obvious. When a voice assistant cuts in early, it comes across as a glitch. When an avatar is looking at you and starts talking while you're mid-sentence, it comes across as rude.

What changed with Deepgram Flux

We tested several approaches to speech recognition and turn detection, and Flux came out ahead. It builds end-of-turn detection into the speech model itself, so the call about whether you've finished can use what you said and how you said it, alongside how long you've been quiet.

Compared with the other approach we tested, Flux reduced unwanted barge-ins by around 50% while p95 latency stayed flat.

In a session, you'd notice it in two places. Someone can stop mid-answer to think without the avatar jumping in. And background noise, like a dog barking or a colleague on a call nearby, is much less likely to be treated as the start of a new turn.

Latency was the one thing we weren't willing to trade. Better turn-taking wasn't worth another network hop on every reply, so we run Flux inside our own infrastructure. Going from first tests to self-hosted production took about two weeks.

Deepgram has written up the evaluation from their side in their case study on Anam.

What this means if you're building with Anam

Nobody talking to an Anam avatar is reading from a script. A sales rep practising a pricing objection might need five seconds to think. A candidate in a mock interview is nervous and restarts sentences. Someone learning a language speaks slowly, with long gaps between words. For our customers these are normal sessions, and they shouldn't have to tune their product around them.

If you use Anam's turnkey agents, this is handled for you. Speech recognition, turn detection, interruption handling, voice and face all run in the same pipeline, with nothing extra to configure. Since August, full-pipeline personas can also use Deepgram for transcription as well as turn-taking (it's in the changelog).

If you bring your own agent and use Anam as the face layer, through something like LiveKit, turn detection lives in your stack. Our post on speech detection events covers how to keep the avatar's listening state in sync with it.

Since Flux went live in supported English sessions, we've had fewer complaints about turn-taking. A few customers who were running their own speech-to-text have moved over to the turnkey setup as well.

What we haven't fixed yet

Language coverage is the biggest gap. The improved turn-taking now works in 10 languages, but we want to support hundreds. Turn-taking also isn't something you solve once. People misjudge when someone's finished talking all the time, and a model will still get some pauses wrong. We'll keep measuring it.

The other piece is what the avatar does while it listens. We've started adding small listening sounds, like "mhm" and "yeah", so avatars don't sit in total silence while you talk. You can see some of this in action in our video on more natural responses.

The quickest way to judge all this is to try it. Build a persona in Anam Lab, then pause mid-sentence, or interrupt it, and see what it does.

FAQ

Can users interrupt an Anam avatar?

Yes. Anam supports barge-in: when someone starts speaking, the avatar stops talking and the new input goes back to the agent. With Flux handling turn detection, thinking pauses and background noise trigger far fewer false interruptions.

Does Anam use Deepgram?

Anam uses Deepgram Flux for turn-taking. Full-pipeline personas can use either Anam or Deepgram for transcription.

What is end-of-turn detection?

End-of-turn detection is how a voice agent or AI avatar decides someone has finished speaking and it's time to reply. The simplest version waits for a fixed stretch of silence. Model-based approaches like Deepgram Flux also use what was said and how it was said, so a thinking pause is less likely to be read as the end of a turn.

Why do AI avatars interrupt people?

Most interruptions happen when the system mistakes a pause for the end of a turn. With a short silence timer, someone stopping to think, restarting a sentence, or a dog barking in the background can all trigger a reply too early. A longer timer avoids this, but then every reply feels slow.

How does Anam reduce avatar interruptions without adding latency?

Anam moved turn detection to Deepgram Flux, which cut unwanted barge-ins by around 50% compared with the other approach Anam tested, while p95 latency stayed flat. Anam runs Flux inside its own infrastructure, so it adds no network hop to each reply.

Does Anam's turn-taking work if I bring my own agent?

With Anam's turnkey agents, turn detection is built in and needs no setup. If you bring your own agent, for example through LiveKit, turn detection runs in your stack. Anam's speech detection events keep the avatar's listening state in sync with it.