What the indicators mean
The indicator compares your median (p50) for the selected period against thresholds derived from latency across all production sessions:- Green — at or better than a healthy production median. No action needed.
- Amber — slower than typical. Users will notice the delay; review the suggestions below.
- Red — slower than roughly 90% of production sessions. Something in your configuration or environment is adding significant delay.
| Metric | Measures | Green | Amber | Red |
|---|---|---|---|---|
| Response latency | User stops speaking → persona starts speaking | ≤ 1.5s | 1.5–3s | > 3s |
| Transcription | User stops speaking → transcript complete | ≤ 0.5s | 0.5–1s | > 1s |
| LLM first output | Transcript complete → first LLM output | ≤ 1s | 1–2.5s | > 2.5s |
| TTS first audio | Speech generation start → persona starts speaking | ≤ 0.5s | 0.5–1s | > 1s |
Response latency
Response latency is the end-to-end delay your users experience: the time from the moment they stop speaking to the moment the persona starts speaking. It is the sum of the three stages below, so start by checking which stage is amber or red and follow the guidance for that stage. If all three stages are green but response latency is amber or red, the gap is usually network transit between your users and Anam:- Check where your users are located. Sessions are served from the closest Anam data center, so users far from one will see higher latency.
- Use the per-session analytics
endpoint to find the slow
sessions, then check their
clientMetadataand location for a pattern.
Transcription
Transcription latency is how long it takes to produce the final transcript after the user stops speaking. Anam manages the transcription pipeline, so this stage is normally well under half a second. If it is amber or red:- Review your voice detection settings.
A high
endOfSpeechSensitivitycan delay the point at which Anam decides the user has finished speaking. - Check whether the affected sessions come from users on poor network connections; delayed or dropped audio extends transcription time.
- If it stays red across many sessions with default settings, contact your Anam representative or support — this stage is Anam-managed.
LLM first output
LLM first output is the time from transcript completion to the first token from the language model. This is usually the largest and most controllable stage. If it is amber or red:- Choose a faster model. Larger models take longer to produce their first token. See available LLMs for the options and their trade-offs.
- Shorten your system prompt. Very long prompts increase time to first token. The prompting guide covers how to keep prompts effective and compact.
- If you use a custom LLM, host it close to Anam’s infrastructure, make sure streaming is enabled, and measure your endpoint’s own time to first token — Anam can only be as fast as your server. See custom LLMs.
- Review knowledge and tools. Knowledge retrieval and tool calls run during the response and add to the delay on turns that use them. Check the tool call timings on the session endpoint to see how long each call took.
TTS first audio
TTS first audio is the time from the start of speech generation to the persona actually speaking. This stage is tightly managed by Anam and sits around half a second for almost all production sessions. If it is amber or red:- Try a different voice. Some voices and voice providers generate the first audio chunk faster than others; see voice configuration.
- If it stays red across voices, contact your Anam representative or support — this stage is Anam-managed.
Digging deeper
- Aggregate analytics endpoint — the same percentiles as the Performance card, filterable by persona, API key, client label, and session type, including the slowest individual turns in the period.
- Per-session analytics endpoint — per-turn timing for one session, so you can see exactly which stage was slow on which turn.