> ## Documentation Index
> Fetch the complete documentation index at: https://anam.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Listening for Events

> Listen to events during Anam AI sessions

## Adding Event Listeners

Register event listeners after creating the Anam client and before starting the session with `stream()` or `streamToVideoElement()`.

<Warning>
  Event handlers must be registered before the session starts. Startup events such as microphone permission, connection, stream, and `SESSION_READY` can fire while `stream()` or `streamToVideoElement()` is running, and late listeners will not receive events that already fired. This also applies to `registerToolCallHandler`.
</Warning>

Use the `addListener` method before the start call:

```typescript theme={"system"}
import { createClient, AnamEvent } from "@anam-ai/js-sdk";

const anamClient = createClient(sessionToken);

anamClient.addListener(AnamEvent.CONNECTION_ESTABLISHED, () => {
  console.log("Connection Established");
});

anamClient.addListener(AnamEvent.MESSAGE_HISTORY_UPDATED, (messages) => {
  console.log("Updated Messages:", messages);
});

await anamClient.streamToVideoElement("video-element-id");
```

## Available Events

| Event Name                        | Description                                                                                  | Callback Parameters                                |
| --------------------------------- | -------------------------------------------------------------------------------------------- | -------------------------------------------------- |
| `CONNECTION_ESTABLISHED`          | Fired when the WebRTC connection is established                                              | None                                               |
| `CONNECTION_CLOSED`               | Fired when the connection is terminated                                                      | `reason: ConnectionClosedCode`, `details?: string` |
| `SESSION_READY`                   | Fired after the session initializes and backend components are ready                         | `sessionId: string`                                |
| `VIDEO_STREAM_STARTED`            | Fired when the video stream becomes available                                                | `videoStream: MediaStream`                         |
| `VIDEO_PLAY_STARTED`              | Fired when the first video frames start playing                                              | None                                               |
| `AUDIO_STREAM_STARTED`            | Fired when the audio stream becomes available                                                | `audioStream: MediaStream`                         |
| `INPUT_AUDIO_STREAM_STARTED`      | Fired when microphone input is initialized                                                   | `audioStream: MediaStream`                         |
| `USER_SPEECH_STARTED`             | The user started speaking. No transcript yet — useful for showing a listening indicator.     | `correlationId: string`                            |
| `USER_SPEECH_ENDED`               | The user stopped speaking. A transcript event may follow in `MESSAGE_STREAM_EVENT_RECEIVED`. | `correlationId: string`                            |
| `MESSAGE_HISTORY_UPDATED`         | Fired when a participant finishes speaking with full history                                 | `messages: Message[]`                              |
| `MESSAGE_STREAM_EVENT_RECEIVED`   | Fired with real-time transcription updates                                                   | `event: MessageStreamEvent`                        |
| `DIRECTOR_NOTE_CUE_APPLIED`       | Fired when a Director Notes cue is applied to the avatar                                     | `event: DirectorNoteCueAppliedEvent`               |
| `TALK_STREAM_INTERRUPTED`         | Fired when a user interrupts a `TalkMessageStream`                                           | `correlationId: string`                            |
| `TOOL_CALL_STARTED`               | Fired when any tool call begins                                                              | `event: ToolCallStartedPayload`                    |
| `TOOL_CALL_COMPLETED`             | Fired when a tool call completes successfully                                                | `event: ToolCallCompletedPayload`                  |
| `TOOL_CALL_FAILED`                | Fired when a tool call fails                                                                 | `event: ToolCallFailedPayload`                     |
| `CLIENT_TOOL_EVENT_RECEIVED`      | **Deprecated** - Use `TOOL_CALL_STARTED` instead                                             | `event: ClientToolEvent`                           |
| `SERVER_WARNING`                  | Fired when the server sends a warning message                                                | `message: string`                                  |
| `MIC_PERMISSION_PENDING`          | Fired when microphone permission is being requested                                          | None                                               |
| `MIC_PERMISSION_GRANTED`          | Fired when microphone permission is granted                                                  | None                                               |
| `MIC_PERMISSION_DENIED`           | Fired when microphone permission is denied                                                   | `error: string`                                    |
| `INPUT_AUDIO_DEVICE_CHANGED`      | Fired when the input audio device changes                                                    | `deviceId: string`                                 |
| `REASONING_HISTORY_UPDATED`       | Fired when reasoning/thought history is updated                                              | `messages: ReasoningMessage[]`                     |
| `REASONING_STREAM_EVENT_RECEIVED` | Fired with real-time reasoning updates                                                       | `event: ReasoningStreamEvent`                      |

## Type Definitions

### Message

```typescript theme={"system"}
interface Message {
  id: string;
  content: string;
  role: MessageRole; // 'user' | 'persona'
  interrupted?: boolean;
}
```

### MessageStreamEvent

```typescript theme={"system"}
interface MessageStreamEvent {
  id: string;
  content: string;
  role: MessageRole; // 'user' | 'persona'
  endOfSpeech: boolean;
  interrupted: boolean;
  cueTag?: string; // Director Notes cue that takes effect from the start of this text chunk
}
```

### DirectorNoteCueAppliedEvent

```typescript theme={"system"}
interface DirectorNoteCueAppliedEvent {
  cueTag: string;
  correlationId: string; // Correlation ID for the associated user action
}
```

### ToolCallStartedPayload

```typescript theme={"system"}
interface ToolCallStartedPayload {
  eventUid: string;                 // Unique ID for this event
  toolCallId: string;               // ID of the tool call
  sessionId: string;                // ID of the active session
  toolName: string;                 // The tool name (e.g., "navigate_to_page")
  toolType: string;                 // Tool type (e.g., "client", "server")
  toolSubtype?: string;             // Tool subtype for server events (e.g., "webhook", "knowledge")
  arguments: Record<string, any>;   // LLM-generated parameters
  timestamp: string;                // ISO timestamp
  timestampUserAction: string;      // Timestamp of the user action that triggered the tool call
  userActionCorrelationId: string;  // Id of the user action that triggered the tool call
}
```

### ToolCallCompletedPayload

```typescript theme={"system"}
interface ToolCallCompletedPayload {
  eventUid: string;                 // Unique ID for this event
  toolCallId: string;               // ID of the tool call
  sessionId: string;                // ID of the active session
  toolName: string;                 // The tool name
  toolType: string;                 // Tool type
  toolSubtype?: string;             // Tool subtype
  result: any;                      // Result returned by the handler
  executionTime: number;            // Execution time in milliseconds
  timestamp: string;                // ISO timestamp
  documentsAccessed?: string[];     // List of file names accessed during the tool call, if applicable (Knowledge tools only)
  timestampUserAction: string;      // Timestamp of the user action that triggered the tool call
  userActionCorrelationId: string;  // Id of the user action that triggered the tool call
}
```

### ToolCallFailedPayload

```typescript theme={"system"}
interface ToolCallFailedPayload {
  eventUid: string;                 // Unique ID for this event
  toolCallId: string;               // ID of the tool call
  sessionId: string;                // ID of the active session
  toolName: string;                 // The tool name
  toolType: string;                 // Tool type
  toolSubtype?: string;             // Tool subtype
  errorMessage: string;             // Error description
  executionTime: number;            // Execution time in milliseconds
  timestamp: string;                // ISO timestamp
  timestampUserAction: string;      // Timestamp of the user action that triggered the tool call
  userActionCorrelationId: string;  // Id of the user action that triggered the tool call
}
```

### ToolCallHandler

```typescript theme={"system"}
interface ToolCallHandler {
  onStart?: (payload: ToolCallStartedPayload) => Promise<string | void>;
  onFail?: (payload: ToolCallFailedPayload) => Promise<void>;
  onComplete?: (payload: ToolCallCompletedPayload) => Promise<void>;
}
```

### ClientToolEvent (Deprecated)

<Warning>
  `ClientToolEvent` is deprecated as of SDK v4.9.0. Use `ToolCallStartedPayload` with `registerToolCallHandler` or `TOOL_CALL_STARTED` events instead.
</Warning>

```typescript theme={"system"}
interface ClientToolEvent {
  eventUid: string;           // Unique ID for this event
  sessionId: string;          // Session ID
  eventName: string;          // The tool name (e.g., "navigate_to_page")
  eventData: Record<string, any>; // LLM-generated parameters
  timestamp: string;          // ISO timestamp when event was created
  timestampUserAction: string; // ISO timestamp of user action that triggered this
  userActionCorrelationId: string; // Correlation ID for tracking
}
```

### ConnectionClosedCode

```typescript theme={"system"}
enum ConnectionClosedCode {
  NORMAL = 'CONNECTION_CLOSED_CODE_NORMAL',
  MICROPHONE_PERMISSION_DENIED = 'CONNECTION_CLOSED_CODE_MICROPHONE_PERMISSION_DENIED',
  SIGNALLING_CLIENT_CONNECTION_FAILURE = 'CONNECTION_CLOSED_CODE_SIGNALLING_CLIENT_CONNECTION_FAILURE',
  WEBRTC_FAILURE = 'CONNECTION_CLOSED_CODE_WEBRTC_FAILURE',
  SERVER_CLOSED_CONNECTION = 'CONNECTION_CLOSED_CODE_SERVER_CLOSED_CONNECTION',
}
```

### ReasoningMessage

```typescript theme={"system"}
interface ReasoningMessage {
  id: string;
  content: string;
  role: string;
}
```

### ReasoningStreamEvent

```typescript theme={"system"}
interface ReasoningStreamEvent {
  id: string;
  content: string;
  endOfThought: boolean;
  role: string;
}
```

## Example Usage

### Loading States

Use connection events to manage loading states:

```typescript theme={"system"}
anamClient.addListener(AnamEvent.CONNECTION_ESTABLISHED, () => {
  setIsConnecting(false);
});

anamClient.addListener(AnamEvent.SESSION_READY, (sessionId: string) => {
  console.log("Session ready:", sessionId);
  setIsLoading(false);
});
```

### Connection Closed Handling

Handle connection closures with reason codes:

```typescript theme={"system"}
import { AnamEvent, ConnectionClosedCode } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.CONNECTION_CLOSED,
  (reason: ConnectionClosedCode, details?: string) => {
    switch (reason) {
      case ConnectionClosedCode.NORMAL:
        console.log("Connection closed normally");
        break;
      case ConnectionClosedCode.MICROPHONE_PERMISSION_DENIED:
        showError("Microphone access is required");
        break;
      case ConnectionClosedCode.WEBRTC_FAILURE:
        showError("Connection failed. Please check your network.");
        break;
      default:
        console.log("Connection closed:", reason, details);
    }
  }
);
```

### Microphone Permission Flow

Track microphone permission state:

```typescript theme={"system"}
anamClient.addListener(AnamEvent.MIC_PERMISSION_PENDING, () => {
  showPermissionPrompt("Please allow microphone access");
});

anamClient.addListener(AnamEvent.MIC_PERMISSION_GRANTED, () => {
  hidePermissionPrompt();
  showMicrophoneIndicator();
});

anamClient.addListener(AnamEvent.MIC_PERMISSION_DENIED, (error: string) => {
  showError(`Microphone access denied: ${error}`);
});
```

### Speech detection

Know when the user starts and stops speaking, before any transcript is available:

```typescript theme={"system"}
anamClient.addListener(AnamEvent.USER_SPEECH_STARTED, (correlationId: string) => {
  setIsUserSpeaking(true);
  showListeningIndicator();
});

anamClient.addListener(AnamEvent.USER_SPEECH_ENDED, (correlationId: string) => {
  setIsUserSpeaking(false);
  showProcessingIndicator();
});
```

These fire as soon as the server-side VAD picks up voice activity, before the full transcript arrives via `MESSAGE_STREAM_EVENT_RECEIVED`. The `correlationId` ties the speech start/end pair to the eventual transcript.

If `USER_SPEECH_ENDED` doesn't arrive within \~10 seconds of a `USER_SPEECH_STARTED`, treat it as a dropped detection and reset your UI. This can happen if the connection drops mid-speech.

### Message History

Track conversation history:

```typescript theme={"system"}
import { Message } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.MESSAGE_HISTORY_UPDATED,
  (messages: Message[]) => {
    setConversationHistory(messages);
  }
);
```

### Real-time Transcription

Monitor speech in real-time:

```typescript theme={"system"}
import { MessageStreamEvent, MessageRole } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.MESSAGE_STREAM_EVENT_RECEIVED,
  (event: MessageStreamEvent) => {
    if (event.role === MessageRole.PERSONA) {
      updatePersonaSpeech(event.content);
    } else {
      updateUserSpeech(event.content);
    }

    if (event.endOfSpeech) {
      finalizeSpeech(event.id);
    }
  }
);
```

### Director Notes Cues

Listen for `DIRECTOR_NOTE_CUE_APPLIED` when your application needs to react after Anam confirms that a [Director Notes](/personas/director-notes) cue was applied to the avatar.

<Note>
  Director Notes cue events require `@anam-ai/js-sdk` version 4.20.0 or later.
</Note>

```typescript theme={"system"}
import { AnamEvent, DirectorNoteCueAppliedEvent } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.DIRECTOR_NOTE_CUE_APPLIED,
  ({ cueTag, correlationId }: DirectorNoteCueAppliedEvent) => {
    showCueIndicator(cueTag);
    console.log(`Applied cue ${cueTag}`, { correlationId });
  }
);
```

Use `correlationId` to associate the cue with other events from the same user action.

Each `MESSAGE_STREAM_EVENT_RECEIVED` payload also includes an optional `cueTag` when a Director Notes cue takes effect from the start of that text chunk. If the field is absent, no new cue takes effect at that chunk boundary. Cue boundaries can split persona speech into multiple streamed text chunks, so append `event.content` in arrival order instead of assuming one event per response.

`MESSAGE_HISTORY_UPDATED` is unchanged. Its `Message` items do not include cue metadata.

### Tool Call Handlers (Recommended)

Use `registerToolCallHandler` to register handlers for specific tools before starting the session. This automatically emits `completed` or `failed` events when the handler completes.

```typescript theme={"system"}
import { createClient } from "@anam-ai/js-sdk";

const anamClient = createClient(sessionToken);

const cancelNavHandler = anamClient.registerToolCallHandler("navigate_to_page", {
  onStart: async (payload) => {
    const { page, section } = payload.arguments;
    router.push(`/${page}${section ? `#${section}` : ""}`);
  },
  onComplete: async (payload) => {
    console.log(`tool call completed ${payload.toolName}`);
  },
  onFail: async (payload) => {
    console.log(`tool call failed ${payload.toolName}`);
  },
});

// you can also register partial handlers
const cancelModalHandler = anamClient.registerToolCallHandler("open_modal", {
  onStart: async (payload) => {
    openModal(payload.arguments.modalType, payload.arguments.data);
    return `Opened ${payload.arguments.modalType} modal`;
  },
});

// start the session after handlers are attached
await anamClient.streamToVideoElement("video-element-id");

// unsubscribe when no longer needed
cancelNavHandler();
cancelModalHandler();
```

<Tip>
  `registerToolCallHandler` is the recommended approach for client tools. Register tool call handlers before `stream()` or `streamToVideoElement()` so the persona cannot invoke a tool before your handler is attached.
</Tip>

### Tool Call Events

Listen for tool call lifecycle events across all tool types — client, webhook, and knowledge tools all emit these events, so you can use them for logging, analytics, or monitoring:

```typescript theme={"system"}
import { AnamEvent, ToolCallStartedPayload, ToolCallCompletedPayload, ToolCallFailedPayload } from "@anam-ai/js-sdk";

anamClient.addListener(
  AnamEvent.TOOL_CALL_STARTED,
  (event: ToolCallStartedPayload) => {
    console.log(`Tool started: ${event.toolName} (${event.toolType})`, event.arguments);
  }
);

anamClient.addListener(
  AnamEvent.TOOL_CALL_COMPLETED,
  (event: ToolCallCompletedPayload) => {
    console.log(`Tool completed: ${event.toolName} in ${event.executionTime}ms`);
  }
);

anamClient.addListener(
  AnamEvent.TOOL_CALL_FAILED,
  (event: ToolCallFailedPayload) => {
    console.error(`Tool failed: ${event.toolName} - ${event.errorMessage}`);
  }
);
```

Use `toolType` and `toolSubtype` to distinguish between tool types in your listeners:

```typescript theme={"system"}
anamClient.addListener(AnamEvent.TOOL_CALL_COMPLETED, (event) => {
  switch (event.toolType) {
    case "client":
      console.log(`Client action completed: ${event.toolName}`);
      break;
    case "server":
      if (event.toolSubtype === "webhook") {
        console.log(`Webhook responded: ${event.toolName} in ${event.executionTime}ms`);
      } else if (event.toolSubtype === "knowledge") {
        console.log(`Knowledge search completed: ${event.toolName}`);
      }
      break;
  }
});
```

<Tip>
  These events fire for all tool types. For guides on configuring each type, see [Client Tools](/personas/tools/client-tools), [Webhook Tools](/personas/tools/webhook-tools), and [Knowledge Tools](/personas/knowledge/tools).
</Tip>

### Server Warnings

Handle server-side warnings:

```typescript theme={"system"}
anamClient.addListener(AnamEvent.SERVER_WARNING, (message: string) => {
  console.warn("Server warning:", message);
  showWarningToast(message);
});
```

### Reasoning Events (Extended Thinking)

Track AI reasoning when using models with extended thinking:

```typescript theme={"system"}
import { ReasoningMessage, ReasoningStreamEvent } from "@anam-ai/js-sdk";

// Get complete reasoning history
anamClient.addListener(
  AnamEvent.REASONING_HISTORY_UPDATED,
  (messages: ReasoningMessage[]) => {
    setReasoningHistory(messages);
  }
);

// Stream reasoning in real-time
anamClient.addListener(
  AnamEvent.REASONING_STREAM_EVENT_RECEIVED,
  (event: ReasoningStreamEvent) => {
    updateReasoningDisplay(event.content);
    if (event.endOfThought) {
      finalizeThought(event.id);
    }
  }
);
```

## Removing Event Listeners

Remove listeners to prevent memory leaks, especially in single-page applications:

```typescript theme={"system"}
const handleMessages = (messages: Message[]) => {
  setConversationHistory(messages);
};

// Add listener
anamClient.addListener(AnamEvent.MESSAGE_HISTORY_UPDATED, handleMessages);

// Remove listener when done
anamClient.removeListener(AnamEvent.MESSAGE_HISTORY_UPDATED, handleMessages);
```

### React Example

```typescript theme={"system"}
useEffect(() => {
  const cancelNav = client.registerToolCallHandler("navigate_to_page", {
    onStart: async (payload) => {
      router.push(`/${payload.arguments.page}`);
    },
  });

  const cancelModal = client.registerToolCallHandler("open_modal", {
    onStart: async (payload) => {
      openModal(payload.arguments.modalType);
      return "Modal opened";
    },
  });

  return () => {
    cancelNav();
    cancelModal();
  };
}, [client]);
```

## Learn More

<CardGroup cols={2}>
  <Card title="Client Tools Guide" icon="browser" href="/personas/tools/client-tools">
    Guide with examples for navigation, modals, UI updates, and more
  </Card>

  <Card title="Tools Overview" icon="wrench" href="/personas/tools/overview">
    Learn about all tool types: client, webhook, and knowledge tools
  </Card>
</CardGroup>