Listening for Events

Adding Event Listeners

Register event listeners after creating the Anam client and before starting the session with stream() or streamToVideoElement().

Event handlers must be registered before the session starts. Startup events such as microphone permission, connection, stream, and SESSION_READY can fire while stream() or streamToVideoElement() is running, and late listeners will not receive events that already fired. This also applies to registerToolCallHandler.

Use the addListener method before the start call:

import { createClient, AnamEvent } from "@anam-ai/js-sdk";

const anamClient = createClient(sessionToken);

anamClient.addListener(AnamEvent.CONNECTION_ESTABLISHED, () => {
  console.log("Connection Established");
});

anamClient.addListener(AnamEvent.MESSAGE_HISTORY_UPDATED, (messages) => {
  console.log("Updated Messages:", messages);
});

await anamClient.streamToVideoElement("video-element-id");

Available Events

Event Name	Description	Callback Parameters
`CONNECTION_ESTABLISHED`	Fired when the WebRTC connection is established	None
`CONNECTION_CLOSED`	Fired when the connection is terminated	`reason: ConnectionClosedCode`, `details?: string`
`SESSION_READY`	Fired after the session initializes and backend components are ready	`sessionId: string`
`VIDEO_STREAM_STARTED`	Fired when the video stream becomes available	`videoStream: MediaStream`
`VIDEO_PLAY_STARTED`	Fired when the first video frames start playing	None
`AUDIO_STREAM_STARTED`	Fired when the audio stream becomes available	`audioStream: MediaStream`
`INPUT_AUDIO_STREAM_STARTED`	Fired when microphone input is initialized	`audioStream: MediaStream`
`USER_SPEECH_STARTED`	The user started speaking. No transcript yet — useful for showing a listening indicator.	`correlationId: string`
`USER_SPEECH_ENDED`	The user stopped speaking. A transcript event may follow in `MESSAGE_STREAM_EVENT_RECEIVED`.	`correlationId: string`
`MESSAGE_HISTORY_UPDATED`	Fired when a participant finishes speaking with full history	`messages: Message[]`
`MESSAGE_STREAM_EVENT_RECEIVED`	Fired with real-time transcription updates	`event: MessageStreamEvent`
`DIRECTOR_NOTE_CUE_APPLIED`	Fired when a Director Notes cue is applied to the avatar	`event: DirectorNoteCueAppliedEvent`
`TALK_STREAM_INTERRUPTED`	Fired when a user interrupts a `TalkMessageStream`	`correlationId: string`
`TOOL_CALL_STARTED`	Fired when any tool call begins	`event: ToolCallStartedPayload`
`TOOL_CALL_COMPLETED`	Fired when a tool call completes successfully	`event: ToolCallCompletedPayload`
`TOOL_CALL_FAILED`	Fired when a tool call fails	`event: ToolCallFailedPayload`
`CLIENT_TOOL_EVENT_RECEIVED`	Deprecated - Use `TOOL_CALL_STARTED` instead	`event: ClientToolEvent`
`SERVER_WARNING`	Fired when the server sends a warning message	`message: string`
`MIC_PERMISSION_PENDING`	Fired when microphone permission is being requested	None
`MIC_PERMISSION_GRANTED`	Fired when microphone permission is granted	None
`MIC_PERMISSION_DENIED`	Fired when microphone permission is denied	`error: string`
`INPUT_AUDIO_DEVICE_CHANGED`	Fired when the input audio device changes	`deviceId: string`
`REASONING_HISTORY_UPDATED`	Fired when reasoning/thought history is updated	`messages: ReasoningMessage[]`
`REASONING_STREAM_EVENT_RECEIVED`	Fired with real-time reasoning updates	`event: ReasoningStreamEvent`

Type Definitions

Message

interface Message {
  id: string;
  content: string;
  role: MessageRole; // 'user' | 'persona'
  interrupted?: boolean;
}

MessageStreamEvent

interface MessageStreamEvent {
  id: string;
  content: string;
  role: MessageRole; // 'user' | 'persona'
  endOfSpeech: boolean;
  interrupted: boolean;
  cueTag?: string; // Director Notes cue that takes effect from the start of this text chunk
}

DirectorNoteCueAppliedEvent

interface DirectorNoteCueAppliedEvent {
  cueTag: string;
  correlationId: string; // Correlation ID for the associated user action
}

ToolCallStartedPayload

interface ToolCallStartedPayload {
  eventUid: string;                 // Unique ID for this event
  toolCallId: string;               // ID of the tool call
  sessionId: string;                // ID of the active session
  toolName: string;                 // The tool name (e.g., "navigate_to_page")
  toolType: string;                 // Tool type (e.g., "client", "server")
  toolSubtype?: string;             // Tool subtype for server events (e.g., "webhook", "knowledge")
  arguments: Record<string, any>;   // LLM-generated parameters
  timestamp: string;                // ISO timestamp
  timestampUserAction: string;      // Timestamp of the user action that triggered the tool call
  userActionCorrelationId: string;  // Id of the user action that triggered the tool call
}

ToolCallCompletedPayload

interface ToolCallCompletedPayload {
  eventUid: string;                 // Unique ID for this event
  toolCallId: string;               // ID of the tool call
  sessionId: string;                // ID of the active session
  toolName: string;                 // The tool name
  toolType: string;                 // Tool type
  toolSubtype?: string;             // Tool subtype
  result: any;                      // Result returned by the handler
  executionTime: number;            // Execution time in milliseconds
  timestamp: string;                // ISO timestamp
  documentsAccessed?: string[];     // List of file names accessed during the tool call, if applicable (Knowledge tools only)
  timestampUserAction: string;      // Timestamp of the user action that triggered the tool call
  userActionCorrelationId: string;  // Id of the user action that triggered the tool call
}

ToolCallFailedPayload

interface ToolCallFailedPayload {
  eventUid: string;                 // Unique ID for this event
  toolCallId: string;               // ID of the tool call
  sessionId: string;                // ID of the active session
  toolName: string;                 // The tool name
  toolType: string;                 // Tool type
  toolSubtype?: string;             // Tool subtype
  errorMessage: string;             // Error description
  executionTime: number;            // Execution time in milliseconds
  timestamp: string;                // ISO timestamp
  timestampUserAction: string;      // Timestamp of the user action that triggered the tool call
  userActionCorrelationId: string;  // Id of the user action that triggered the tool call
}

ToolCallHandler

interface ToolCallHandler {
  onStart?: (payload: ToolCallStartedPayload) => Promise<string | void>;
  onFail?: (payload: ToolCallFailedPayload) => Promise<void>;
  onComplete?: (payload: ToolCallCompletedPayload) => Promise<void>;
}

ClientToolEvent (Deprecated)

ClientToolEvent is deprecated as of SDK v4.9.0. Use ToolCallStartedPayload with registerToolCallHandler or TOOL_CALL_STARTED events instead.

interface ClientToolEvent {
  eventUid: string;           // Unique ID for this event
  sessionId: string;          // Session ID
  eventName: string;          // The tool name (e.g., "navigate_to_page")
  eventData: Record<string, any>; // LLM-generated parameters
  timestamp: string;          // ISO timestamp when event was created
  timestampUserAction: string; // ISO timestamp of user action that triggered this
  userActionCorrelationId: string; // Correlation ID for tracking
}

ConnectionClosedCode

enum ConnectionClosedCode {
  NORMAL = 'CONNECTION_CLOSED_CODE_NORMAL',
  MICROPHONE_PERMISSION_DENIED = 'CONNECTION_CLOSED_CODE_MICROPHONE_PERMISSION_DENIED',
  SIGNALLING_CLIENT_CONNECTION_FAILURE = 'CONNECTION_CLOSED_CODE_SIGNALLING_CLIENT_CONNECTION_FAILURE',
  WEBRTC_FAILURE = 'CONNECTION_CLOSED_CODE_WEBRTC_FAILURE',
  SERVER_CLOSED_CONNECTION = 'CONNECTION_CLOSED_CODE_SERVER_CLOSED_CONNECTION',
}

ReasoningMessage

interface ReasoningMessage {
  id: string;
  content: string;
  role: string;
}

ReasoningStreamEvent

interface ReasoningStreamEvent {
  id: string;
  content: string;
  endOfThought: boolean;
  role: string;
}

Example Usage

Loading States

Use connection events to manage loading states:

anamClient.addListener(AnamEvent.CONNECTION_ESTABLISHED, () => {
  setIsConnecting(false);
});

anamClient.addListener(AnamEvent.SESSION_READY, (sessionId: string) => {
  console.log("Session ready:", sessionId);
  setIsLoading(false);
});

Connection Closed Handling

Handle connection closures with reason codes:

import { AnamEvent, ConnectionClosedCode } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.CONNECTION_CLOSED,
  (reason: ConnectionClosedCode, details?: string) => {
    switch (reason) {
      case ConnectionClosedCode.NORMAL:
        console.log("Connection closed normally");
        break;
      case ConnectionClosedCode.MICROPHONE_PERMISSION_DENIED:
        showError("Microphone access is required");
        break;
      case ConnectionClosedCode.WEBRTC_FAILURE:
        showError("Connection failed. Please check your network.");
        break;
      default:
        console.log("Connection closed:", reason, details);
    }
  }
);

Microphone Permission Flow

Track microphone permission state:

anamClient.addListener(AnamEvent.MIC_PERMISSION_PENDING, () => {
  showPermissionPrompt("Please allow microphone access");
});

anamClient.addListener(AnamEvent.MIC_PERMISSION_GRANTED, () => {
  hidePermissionPrompt();
  showMicrophoneIndicator();
});

anamClient.addListener(AnamEvent.MIC_PERMISSION_DENIED, (error: string) => {
  showError(`Microphone access denied: ${error}`);
});

Speech detection

Know when the user starts and stops speaking, before any transcript is available:

anamClient.addListener(AnamEvent.USER_SPEECH_STARTED, (correlationId: string) => {
  setIsUserSpeaking(true);
  showListeningIndicator();
});

anamClient.addListener(AnamEvent.USER_SPEECH_ENDED, (correlationId: string) => {
  setIsUserSpeaking(false);
  showProcessingIndicator();
});

These fire as soon as the server-side VAD picks up voice activity, before the full transcript arrives via MESSAGE_STREAM_EVENT_RECEIVED. The correlationId ties the speech start/end pair to the eventual transcript. If USER_SPEECH_ENDED doesn’t arrive within ~10 seconds of a USER_SPEECH_STARTED, treat it as a dropped detection and reset your UI. This can happen if the connection drops mid-speech.

Message History

Track conversation history:

import { Message } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.MESSAGE_HISTORY_UPDATED,
  (messages: Message[]) => {
    setConversationHistory(messages);
  }
);

Real-time Transcription

Monitor speech in real-time:

import { MessageStreamEvent, MessageRole } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.MESSAGE_STREAM_EVENT_RECEIVED,
  (event: MessageStreamEvent) => {
    if (event.role === MessageRole.PERSONA) {
      updatePersonaSpeech(event.content);
    } else {
      updateUserSpeech(event.content);
    }

    if (event.endOfSpeech) {
      finalizeSpeech(event.id);
    }
  }
);

Director Notes Cues

Listen for DIRECTOR_NOTE_CUE_APPLIED when your application needs to react after Anam confirms that a Director Notes cue was applied to the avatar.

Director Notes cue events require @anam-ai/js-sdk version 4.20.0 or later.

import { AnamEvent, DirectorNoteCueAppliedEvent } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.DIRECTOR_NOTE_CUE_APPLIED,
  ({ cueTag, correlationId }: DirectorNoteCueAppliedEvent) => {
    showCueIndicator(cueTag);
    console.log(`Applied cue ${cueTag}`, { correlationId });
  }
);

Use correlationId to associate the cue with other events from the same user action. Each MESSAGE_STREAM_EVENT_RECEIVED payload also includes an optional cueTag when a Director Notes cue takes effect from the start of that text chunk. If the field is absent, no new cue takes effect at that chunk boundary. Cue boundaries can split persona speech into multiple streamed text chunks, so append event.content in arrival order instead of assuming one event per response. MESSAGE_HISTORY_UPDATED is unchanged. Its Message items do not include cue metadata.

Tool Call Handlers (Recommended)

Use registerToolCallHandler to register handlers for specific tools before starting the session. This automatically emits completed or failed events when the handler completes.

import { createClient } from "@anam-ai/js-sdk";

const anamClient = createClient(sessionToken);

const cancelNavHandler = anamClient.registerToolCallHandler("navigate_to_page", {
  onStart: async (payload) => {
    const { page, section } = payload.arguments;
    router.push(`/${page}${section ? `#${section}` : ""}`);
  },
  onComplete: async (payload) => {
    console.log(`tool call completed ${payload.toolName}`);
  },
  onFail: async (payload) => {
    console.log(`tool call failed ${payload.toolName}`);
  },
});

// you can also register partial handlers
const cancelModalHandler = anamClient.registerToolCallHandler("open_modal", {
  onStart: async (payload) => {
    openModal(payload.arguments.modalType, payload.arguments.data);
    return `Opened ${payload.arguments.modalType} modal`;
  },
});

// start the session after handlers are attached
await anamClient.streamToVideoElement("video-element-id");

// unsubscribe when no longer needed
cancelNavHandler();
cancelModalHandler();

registerToolCallHandler is the recommended approach for client tools. Register tool call handlers before stream() or streamToVideoElement() so the persona cannot invoke a tool before your handler is attached.

Tool Call Events

Listen for tool call lifecycle events across all tool types — client, webhook, and knowledge tools all emit these events, so you can use them for logging, analytics, or monitoring:

import { AnamEvent, ToolCallStartedPayload, ToolCallCompletedPayload, ToolCallFailedPayload } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.TOOL_CALL_STARTED,
  (event: ToolCallStartedPayload) => {
    console.log(`Tool started: ${event.toolName} (${event.toolType})`, event.arguments);
  }
);

anamClient.addListener(
  AnamEvent.TOOL_CALL_COMPLETED,
  (event: ToolCallCompletedPayload) => {
    console.log(`Tool completed: ${event.toolName} in ${event.executionTime}ms`);
  }
);

anamClient.addListener(
  AnamEvent.TOOL_CALL_FAILED,
  (event: ToolCallFailedPayload) => {
    console.error(`Tool failed: ${event.toolName} - ${event.errorMessage}`);
  }
);

Use toolType and toolSubtype to distinguish between tool types in your listeners:

anamClient.addListener(AnamEvent.TOOL_CALL_COMPLETED, (event) => {
  switch (event.toolType) {
    case "client":
      console.log(`Client action completed: ${event.toolName}`);
      break;
    case "server":
      if (event.toolSubtype === "webhook") {
        console.log(`Webhook responded: ${event.toolName} in ${event.executionTime}ms`);
      } else if (event.toolSubtype === "knowledge") {
        console.log(`Knowledge search completed: ${event.toolName}`);
      }
      break;
  }
});

These events fire for all tool types. For guides on configuring each type, see Client Tools, Webhook Tools, and Knowledge Tools.

Server Warnings

Handle server-side warnings:

anamClient.addListener(AnamEvent.SERVER_WARNING, (message: string) => {
  console.warn("Server warning:", message);
  showWarningToast(message);
});

Reasoning Events (Extended Thinking)

Track AI reasoning when using models with extended thinking:

import { ReasoningMessage, ReasoningStreamEvent } from "@anam-ai/js-sdk";

// Get complete reasoning history
anamClient.addListener(
  AnamEvent.REASONING_HISTORY_UPDATED,
  (messages: ReasoningMessage[]) => {
    setReasoningHistory(messages);
  }
);

// Stream reasoning in real-time
anamClient.addListener(
  AnamEvent.REASONING_STREAM_EVENT_RECEIVED,
  (event: ReasoningStreamEvent) => {
    updateReasoningDisplay(event.content);
    if (event.endOfThought) {
      finalizeThought(event.id);
    }
  }
);

Removing Event Listeners

Remove listeners to prevent memory leaks, especially in single-page applications:

const handleMessages = (messages: Message[]) => {
  setConversationHistory(messages);
};

// Add listener
anamClient.addListener(AnamEvent.MESSAGE_HISTORY_UPDATED, handleMessages);

// Remove listener when done
anamClient.removeListener(AnamEvent.MESSAGE_HISTORY_UPDATED, handleMessages);

React Example

useEffect(() => {
  const cancelNav = client.registerToolCallHandler("navigate_to_page", {
    onStart: async (payload) => {
      router.push(`/${payload.arguments.page}`);
    },
  });

  const cancelModal = client.registerToolCallHandler("open_modal", {
    onStart: async (payload) => {
      openModal(payload.arguments.modalType);
      return "Modal opened";
    },
  });

  return () => {
    cancelNav();
    cancelModal();
  };
}, [client]);

Getting Started

SDK Reference

Examples

Listening for Events

Adding Event Listeners

Available Events

Type Definitions

Message

MessageStreamEvent

DirectorNoteCueAppliedEvent

ToolCallStartedPayload

ToolCallCompletedPayload

ToolCallFailedPayload

ToolCallHandler

ClientToolEvent (Deprecated)

ConnectionClosedCode

ReasoningMessage

ReasoningStreamEvent

Example Usage

Loading States

Connection Closed Handling

Microphone Permission Flow

Speech detection

Message History

Real-time Transcription

Director Notes Cues

Tool Call Handlers (Recommended)

Tool Call Events

Server Warnings

Reasoning Events (Extended Thinking)

Removing Event Listeners

React Example

Learn More

Client Tools Guide

Tools Overview

​Adding Event Listeners

​Available Events

​Type Definitions

​Message

​MessageStreamEvent

​DirectorNoteCueAppliedEvent

​ToolCallStartedPayload

​ToolCallCompletedPayload

​ToolCallFailedPayload

​ToolCallHandler

​ClientToolEvent (Deprecated)

​ConnectionClosedCode

​ReasoningMessage

​ReasoningStreamEvent

​Example Usage

​Loading States

​Connection Closed Handling

​Microphone Permission Flow

​Speech detection

​Message History

​Real-time Transcription

​Director Notes Cues

​Tool Call Handlers (Recommended)

​Tool Call Events

​Server Warnings

​Reasoning Events (Extended Thinking)

​Removing Event Listeners

​React Example

​Learn More

Client Tools Guide

Tools Overview

Adding Event Listeners

Available Events

Type Definitions

Message

MessageStreamEvent

DirectorNoteCueAppliedEvent

ToolCallStartedPayload

ToolCallCompletedPayload

ToolCallFailedPayload

ToolCallHandler

ClientToolEvent (Deprecated)

ConnectionClosedCode

ReasoningMessage

ReasoningStreamEvent

Example Usage

Loading States

Connection Closed Handling

Microphone Permission Flow

Speech detection

Message History

Real-time Transcription

Director Notes Cues

Tool Call Handlers (Recommended)

Tool Call Events

Server Warnings

Reasoning Events (Extended Thinking)

Removing Event Listeners

React Example

Learn More