# Server API Reference
Source: https://docs.spatialreal.ai/api-reference/api-reference

Server-side API Reference

## Obtain a session token

Call the server endpoint `POST /v1/console/session-tokens`.

### Console API host

In the examples below, replace `<console-api-host>` with the host for your region:

| Region         | `<console-api-host>`                     |
| -------------- | ---------------------------------------- |
| `ap-northeast` | `console.ap-northeast.spatialwalk.cloud` |
| `us-west`      | `console.us-west.spatialwalk.cloud`      |

### Request parameters

* **Body**
  * **expireAt**: int, expiration timestamp in seconds. Range: now \< expireAt \< now + 24 hours
  * **modelVersion**: string, reserved field; omit
* **Header**
  * **X-Api-Key**: your API key

### Request example

```bash theme={null}
curl --location --request POST 'https://<console-api-host>/v1/console/session-tokens' \  # replace with your console API domain
--header 'X-Api-Key: <your-api-key>' \                                                  # replace with your API key
--header 'Content-Type: application/json' \
--data-raw '{
    "expireAt": 1754824283,
    "modelVersion": ""
}'                                                                                      # set expireAt to the desired expiration time
```

### Response example

```json theme={null}
{
  "sessionToken": "..."
}
```


# Authentication
Source: https://docs.spatialreal.ai/api-reference/auth

Authentication Flow

## Before you start

Before you begin, make sure you have your app-id and api-key.
AvatarKit authentication requires a server-side component, so you need to implement an authentication endpoint on your own server.

## Connection flow

1. The client sends an authentication request to your business server.
2. The business server sends a request to SpatialWalk's server to generate a session token, including information such as expiration time in the request body, and providing the api-key.
3. The SpatialWalk server returns the session token to your business server.
4. The business server returns the session token to the client.
5. The client initializes AvatarKit with the session token.
6. Inside AvatarKit, a connection request is created to the SpatialWalk server using the session token.

```mermaid theme={null}
sequenceDiagram
    participant Client as Client
    participant BizServer as Business Server
    participant SW as SpatialWalk Server
    participant Kit as AvatarKit

    Client->>BizServer: Authentication request
    BizServer->>SW: Generate session token (with api-key, expiration)
    SW-->>BizServer: Return session token
    BizServer-->>Client: Return session token
    Client->>Kit: Initialize with session token
    Kit->>SW: Create connection request (session token)
    SW-->>Kit: Connection response
    Note over SW,Kit: After token expiration, new connections are rejected<br>Existing connections are not affected
```

## Token expiration

If you attempt to establish a new connection after the token's configured expiration time, it will be rejected.
Existing established connections are not affected.

## Notes

* Avoid leaking your api-key; ensure it is only used on the server.
* The session token is designed to be single-use. Ensure a new token is used for each connection.

> For detailed authentication API docs, see the [API Reference](/api-reference/api-reference)


# Regions
Source: https://docs.spatialreal.ai/api-reference/regions

Available regions, how to configure and how to fallback to other region for higher availability.

## Globally distributed service

Our avatar service runs in multiple regions. You can choose the region that’s closest to your infrastructure to minimize latency.

## Available regions

Use these region names when selecting a deployment:

* **ap-northeast**
* **us-west**

## Console API hosts

When a doc example uses `<console-api-host>`, substitute the host for your region:

| Region         | `<console-api-host>`                     |
| -------------- | ---------------------------------------- |
| `ap-northeast` | `console.ap-northeast.spatialwalk.cloud` |
| `us-west`      | `console.us-west.spatialwalk.cloud`      |

## Console and Ingress endpoint URLs

When a doc example uses console endpoint url or ingress endpoint url, substitute the URL for your region:

| Region         | Console Endpoint URL                                        | Ingress Endpoint URL (v2)                                   |
| -------------- | ----------------------------------------------------------- | ----------------------------------------------------------- |
| `ap-northeast` | `https://console.ap-northeast.spatialwalk.cloud/v1/console` | `wss://api.ap-northeast.spatialwalk.cloud/v2/driveningress` |
| `us-west`      | `https://console.us-west.spatialwalk.cloud/v1/console`      | `wss://api.us-west.spatialwalk.cloud/v2/driveningress`      |

## Cross-region portability

The following identifiers/credentials are **not region-specific** and can be used across regions:

* **App ID**
* **API Key**
* **Avatar ID**
* **Session Token** (generated by the console API)

This means you can switch regions or run multi-region without recreating apps, rotating keys, or re-generating avatars.

## High availability with regional fallback

Services in different regions run on **separate infrastructure**, so you can improve availability by implementing fallback:

* **Primary + backup**: if requests to the primary region fail, retry against another region.
* **Data sync**: since the identifiers/credentials mentioned above are portable, you can keep using them when failing over to another region.


# Audio Specification
Source: https://docs.spatialreal.ai/concepts/audio-spec

Accepted PCM audio format for SpatialReal services.

## Supported Input Audio

SpatialReal services currently accept **mono 16-bit PCM (`s16le`)** audio as raw PCM bytes.

<CardGroup>
  <Card title="Format" icon="waveform">
    Raw PCM bytes in `s16le` format
  </Card>

  <Card title="Channels" icon="audio-lines">
    `1` channel (mono)
  </Card>

  <Card title="Bit Depth" icon="microchip">
    `16-bit` samples
  </Card>

  <Card title="Sample Rates" icon="gauge-high">
    `8000` to `48000` Hz from the supported set below
  </Card>
</CardGroup>

## Specification

| Property     | Value                                                               |
| ------------ | ------------------------------------------------------------------- |
| Sample Rate  | One of `8000`, `16000`, `22050`, `24000`, `32000`, `44100`, `48000` |
| Channels     | `1` (mono)                                                          |
| Bit Depth    | `16-bit`                                                            |
| Format       | Raw PCM bytes                                                       |
| PCM Encoding | `s16le`                                                             |

<Note>
  If your source audio uses stereo channels, floating point samples, compressed codecs, or an unsupported sample rate, convert it before sending it to SpatialReal.
</Note>

## What `s16le` Means

* `s16` means each audio sample is a signed 16-bit integer.
* `le` means the bytes are stored in little-endian order.
* `mono` means the stream contains a single channel only.

## Common Conversion Checklist

Before sending audio to the SDK or service, make sure your pipeline outputs:

* a supported sample rate
* mono audio
* 16-bit signed PCM samples
* raw PCM bytes rather than WAV headers or compressed audio frames

For a practical overview of common audio encodings and container formats, see the [FFmpeg audio types reference](https://trac.ffmpeg.org/wiki/audio%20types).


# Integration Modes
Source: https://docs.spatialreal.ai/concepts/integrations

Choose the right integration approach for your use case

## Choose Your Integration Mode

SpatialReal offers four distinct integration modes to suit different architectural needs, latency requirements, and development preferences.

<CardGroup>
  <Card title="SDK Mode" icon="mobile" href="#sdk-mode">
    Client-centric integration with minimal server-side changes
  </Card>

  <Card title="RTC Mode" icon="tower-broadcast" href="#rtc-mode">
    Ultra-low latency via LiveKit or Agora
  </Card>

  <Card title="Framework Plugin" icon="plug" href="#framework-plugin">
    Seamless integration with LiveKit Agents or TEN Framework
  </Card>

  <Card title="Host Mode" icon="server" href="#host-mode">
    Full control with custom transport layer
  </Card>
</CardGroup>

***

## SDK Mode

In this mode, the client-side application manages the audio input. The developer passes the audio to the SpatialReal Client SDK, which handles the server interaction to retrieve animation data and render the avatar.

<Steps>
  <Step title="Pass Audio">
    Developer passes audio to the SpatialReal SDK on the client.
  </Step>

  <Step title="Inference">
    SDK calls the inference service.
  </Step>

  <Step title="Render">
    SDK receives drive parameters and plays the avatar.
  </Step>
</Steps>

**Best Suited For:**

* **Client-Centric Logic:** Scenarios where the voice agent logic resides primarily on the device.
* **Moderate Latency:** Projects where ultra-low latency is not the absolute priority.
* **Simplified Architecture:** Minimal server-side development required (only for authentication), allowing most logic to remain on the client.

```mermaid actions={false} theme={null}
---
config:
  "look": "handDrawn"
---
flowchart LR
    subgraph Client ["🖥️ Client"]
        App["Client App (Voice Logic)"]
        SDK["SpatialReal SDK"]
    end

    Cloud["☁️ SpatialReal Cloud"]

    App -- "Audio Data" --> SDK
    SDK -- "Send Audio" --> Cloud
    Cloud -- "Drive Params" --> SDK
    SDK -- "Render & Play" --> App

    style Client fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#000
    style Cloud fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#000
```

<Card title="View SDK Mode Guide" icon="arrow-right" href="/guide/sdk-mode" />

***

## RTC Mode

This mode leverages Real-Time Communication (RTC) infrastructure (currently supporting LiveKit and Agora). The developer streams audio to SpatialReal, but instead of returning data to the developer, SpatialReal pushes the drive parameters directly into an RTC room/channel.

<Steps>
  <Step title="Stream Audio">
    Developer streams audio via Server SDK.
  </Step>

  <Step title="Push to RTC">
    SpatialReal Service pushes audio and avatar drive data to an RTC Room.
  </Step>

  <Step title="Subscribe & Play">
    Client joins the room using the SpatialReal RTC Client to subscribe, parse, and play.
  </Step>
</Steps>

<Note>
  The stream contains **binary drive parameters and audio**, not a pre-rendered video feed. Therefore, it cannot be played by standard video players; it requires the SpatialReal Client to render.
</Note>

**Best Suited For:**

* **Existing RTC Users:** Teams already using LiveKit or Agora for voice agents but *not* using their specific agent frameworks (e.g., LiveKit Agents or TEN Framework).
* **Server-Managed State:** Scenarios requiring server-side management of conversation state (e.g., handling interruptions).
* **Ultra-Low Latency:** Leveraging established RTC networks for minimal delay.

```mermaid actions={false} theme={null}
---
config:
  "look": "handDrawn"
---
flowchart BT
    subgraph Server ["🔧 Developer Backend"]
        DevServer["Server Logic"]
        ServerSDK["SpatialReal Server SDK"]
        DevServer --> ServerSDK
    end

    subgraph Cloud ["☁️ SpatialReal Cloud"]
        Service["Avatar Service"]
        RTCPub["RTC Egress"]
        Service --> RTCPub
    end

    subgraph RTC ["📡 RTC Provider"]
        Room["LiveKit Room / Agora Channel"]
    end

    subgraph Client ["🖥️ Client"]
        RTCClient["SpatialReal RTC Client"]
    end

    ServerSDK -->|Audio Stream| Service
    RTCPub -->|Drive Params + Audio| Room
    Room -->|Subscribe| RTCClient

    style Client fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#000
    style RTC fill:#f3e5f5,stroke:#9C27B0,stroke-width:2px,color:#000
    style Cloud fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#000
    style Server fill:#e8eaf6,stroke:#3F51B5,stroke-width:2px,color:#000
```

<Card title="View RTC Mode Guide" icon="arrow-right" href="/guide/rtc-mode" />

***

## Framework Plugin

This is the most streamlined approach for modern voice agent frameworks. Developers use a provided plugin that sits inside the voice agent pipeline (e.g., LiveKit Agents, TEN Framework).

<Steps>
  <Step title="Intercept Audio">
    The Plugin intercepts audio from the agent pipeline.
  </Step>

  <Step title="Send to SpatialReal">
    It sends audio to SpatialReal and configures the RTC push parameters.
  </Step>

  <Step title="Push & Render">
    SpatialReal pushes data to the RTC room (Client side is identical to RTC Mode).
  </Step>
</Steps>

**Best Suited For:**

* **Framework Users:** Teams already using frameworks like LiveKit Agents or TEN Framework.
* **Rapid Integration:** Low implementation cost; the plugin automatically handles signal processing and conversation state (interruption logic).
* **Migration:** Easy to switch if you are currently using other avatar services within these frameworks.

```mermaid actions={false} theme={null}
---
config:
  "look": "handDrawn"
---
flowchart BT
    subgraph Agent ["🤖 Voice Agent"]
        Pipeline["Agent Pipeline (LiveKit / TEN)"]
        Plugin["SpatialReal Plugin"]
    end

    subgraph Cloud ["☁️ SpatialReal Cloud"]
        Service["Inference Service"]
        RTCPub["RTC Egress"]
    end

    subgraph RTC ["📡 RTC Provider"]
        Room["LiveKit Room / Agora Channel"]
    end

    subgraph Client ["🖥️ Client"]
        RTCClient["SpatialReal RTC Client"]
    end

    Pipeline -. Audio .-> Plugin
    Plugin -- Audio & Config --> Service
    Service --> RTCPub
    RTCPub -- RTC Push --> Room
    Room -- Subscribe --> RTCClient

    style Client fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#000
    style RTC fill:#f3e5f5,stroke:#9C27B0,stroke-width:2px,color:#000
    style Cloud fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#000
    style Agent fill:#e8eaf6,stroke:#3F51B5,stroke-width:2px,color:#000
```

<Card title="View Framework Plugin Guide" icon="arrow-right" href="/guide/rtc-mode" />

***

## Host Mode

In Host Mode, the developer acts as the bridge. You use the SpatialReal Server SDK to stream audio to the service and receive streaming drive parameters back. It is then your responsibility to transport this data to the client.

<Steps>
  <Step title="Stream Audio">
    Developer sends streaming audio via Server SDK.
  </Step>

  <Step title="Receive Parameters">
    SpatialReal returns streaming drive parameters to the Developer's Server.
  </Step>

  <Step title="Transport">
    Developer transmits audio and parameters to the Client SDK via a custom transport layer.
  </Step>

  <Step title="Render">
    Client SDK renders the avatar.
  </Step>
</Steps>

<Warning>
  The custom transport layer must ensure data is delivered **without duplication, loss, or disorder**.
</Warning>

**Best Suited For:**

* **Custom Transport:** Teams that already have a reliable, controllable transport layer.
* **Deep Integration:** Developers willing to handle server-side adaptation for maximum control.
* **High Low-Latency Demands:** When you need to optimize the network path manually.

```mermaid actions={false} theme={null}
---
config:
  "look": "handDrawn"
---
flowchart LR
    subgraph Client ["🖥️ Client"]
        ClientSDK["SpatialReal Client SDK"]
    end

    subgraph Server ["🔧 Developer Backend"]
        DevServer["Server Logic"]
        ServerSDK["SpatialReal Server SDK"]
    end

    Cloud["☁️ SpatialReal Cloud"]

    DevServer -- "Audio Stream" --> ServerSDK
    ServerSDK -- "Send Audio" --> Cloud
    Cloud -- "Drive Params" --> ServerSDK
    ServerSDK -- "Animation Stream" --> DevServer
    DevServer == "Custom Transport: Audio + Animation" ==> ClientSDK

    style Client fill:#e8f4fd,stroke:#2196F3,stroke-width:2px,color:#000
    style Server fill:#e8eaf6,stroke:#3F51B5,stroke-width:2px,color:#000
    style Cloud fill:#fff3e0,stroke:#FF9800,stroke-width:2px,color:#000
```

<Card title="View Host Mode Guide" icon="arrow-right" href="/guide/host-mode" />

***

## Still Not Sure?

Check integration overview for a side-by-side comparison and runnable example projects.

<Card title="Open Integrations Overview" icon="arrow-right" href="/overview/integration_overview" />

***

## Want a Quick Taste?

Check this quick start guide to get a simple demo up and running in minutes.

<Card title="Open Quickstart Guide" icon="arrow-right" href="/overview/speech-to-avatar" />

***

## Next Steps

<CardGroup>
  <Card title="SDK Mode Guide" icon="book-open" href="/guide/sdk-mode">
    Get started with the simplest integration approach
  </Card>

  <Card title="Host Mode Guide" icon="book-open" href="/guide/host-mode">
    Learn how to implement custom transport
  </Card>

  <Card title="RTC Mode & Framework Plugin Guide" icon="book-open" href="/guide/rtc-mode">
    Set up real-time communication with LiveKit or Agora
  </Card>

  <Card title="Demo Projects" icon="code" href="/overview/demo-projects">
    Explore sample implementations
  </Card>
</CardGroup>


# Host Mode
Source: https://docs.spatialreal.ai/guide/host-mode

Self-managed networking with AvatarKit Server SDK

## What is Host Mode?

In Host Mode, **your application** manages the network connection to AvatarKit's server-side SDK. Your server sends encoded messages to your client, and the client SDK receives and **decodes them internally** for synchronized playback and rendering.

```mermaid theme={null}
flowchart LR
    A["🖥️ AvatarKit<br/>Server SDK"] -->|Encoded Messages| B["Your App<br/>(Network Layer)"]
    B -->|yieldAudioData| C["AvatarKit<br/>Client SDK"]
    B -->|yieldFramesData| C
    C -->|Decode & Render| D["🖥️ Avatar Rendering"]
```

<Warning>
  Host Mode **requires** AvatarKit's server-side SDK to generate the encoded messages. The data passed to `yieldAudioData()` and `yieldFramesData()` are encoded messages from the server SDK — not raw audio or animation data you create yourself.
</Warning>

## When to Use

* **Custom network layer** — you manage the connection between your client and AvatarKit's server SDK yourself
* **RTC integration** — messages are relayed through a real-time communication server (LiveKit, Agora, etc.)
* **Proxy architecture** — your backend acts as a relay between the client and AvatarKit server

## Requirements

| Requirement              | Description                                                                  |
| ------------------------ | ---------------------------------------------------------------------------- |
| **App ID**               | Obtained from [SpatialReal Studio](https://app.spatialreal.ai)               |
| **Session Token**        | **Not required** on the client side                                          |
| **AvatarKit Server SDK** | Your backend must integrate with AvatarKit's server SDK to generate messages |

## SDK Mode vs Host Mode

| Aspect               | SDK Mode                                         | Host Mode                                          |
| -------------------- | ------------------------------------------------ | -------------------------------------------------- |
| **Network**          | Client SDK connects to AvatarKit server directly | Your app relays messages from AvatarKit Server SDK |
| **Message Decoding** | Handled internally                               | Handled internally (same)                          |
| **Session Token**    | Required (client-side)                           | Not required (client-side)                         |
| **Server SDK**       | Not needed                                       | **Required** on your backend                       |
| **Key Methods**      | `send()`, `start()`, `close()`                   | `yieldAudioData()`, `yieldFramesData()`            |
| **Use Case**         | Simplest integration                             | Custom networking / RTC relay                      |

## Key Concepts

### ConversationId Management

ConversationId links audio and animation messages for a single conversation session:

1. Call `yieldAudioData()` — returns a `conversationId`
2. Use that `conversationId` when calling `yieldFramesData()`
3. Messages with a **mismatched** conversationId will be **discarded**
4. Use `getCurrentConversationId()` to retrieve the current active session ID

<Warning>
  **Important:** Always use the conversationId returned by `yieldAudioData()` when sending animation messages. Mismatched IDs cause messages to be silently dropped.
</Warning>

### Fallback Mechanism

If you provide empty animation data (empty array or undefined), the SDK automatically enters **audio-only mode** for that session. Once in audio-only mode, any subsequent animation data for that session is ignored — only audio continues playing.

## Get Started

<CardGroup>
  <Card title="Web Demo" icon="globe" href="https://github.com/spatialwalk/avatar-kit-web-demo">
    GitHub demo repository
  </Card>

  <Card title="iOS Demo" icon="apple" href="https://github.com/spatialwalk/avatar-kit-ios-demo">
    GitHub demo repository
  </Card>

  <Card title="Android Demo" icon="android" href="https://github.com/spatialwalk/avatar-kit-android-demo">
    GitHub demo repository
  </Card>
</CardGroup>


# Introduction
Source: https://docs.spatialreal.ai/guide/introduction

Step-by-step guides for integrating SpatialReal avatars into your application

The **Guide** walks you through integrating SpatialReal avatars by **integration mode**. Choose the mode that matches your architecture, then follow the linked guides for your platform (Web, iOS, Android) or backend (LiveKit, Server SDK).

## Integration Modes

<CardGroup>
  <Card title="SDK Mode" icon="code" href="/guide/sdk-mode">
    Client sends audio; SDK handles networking and rendering. Best for simple, client-centric apps.
  </Card>

  <Card title="RTC Mode & Framework Plugins" icon="tower-broadcast" href="/guide/rtc-mode">
    Real-time voice via LiveKit. Use framework plugins or Server SDK with egress.
  </Card>

  <Card title="Host Mode" icon="server" href="/guide/host-mode">
    You control the transport. Server SDK sends audio and relays animation to the client SDK.
  </Card>

  <Card title="Agent Mode" icon="robot" href="/guide/agent-mode">
    Fully managed voice agent (coming soon).
  </Card>
</CardGroup>

## Where to Start

* **New to SpatialReal?** Start with [Overview → Speech-to-Avatar Quickstart](/overview/speech-to-avatar), then open the guide for your chosen mode.
* **Already chose a mode?** Use the sidebar: pick **SDK Mode**, **RTC Mode**, or **Host Mode**, then follow the platform-specific quickstarts (Web, iOS, Android) or server guides (LiveKit).
* **Want working code?** Check out our [Demo Projects](/overview/demo-projects) for complete examples you can run immediately.

<Card title="AvatarKit Voice Agent Demo" icon="github" href="https://github.com/spatialwalk/avatarkit-voice-agent-demo">
  A full reference repository with implementation details, including different frontend UI options and multiple backend agent patterns.
</Card>

## Quick Links

| Mode     | Client / Platform                                                                            | Server                                                                                                                    |
| -------- | -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| **SDK**  | [Web](/guide/sdk-mode-web) · [iOS](/ios-sdk/quickstart) · [Android](/android-sdk/quickstart) | Session token from your backend only                                                                                      |
| **RTC**  | [LiveKit Client](/guide/rtc-livekit-client)                                                  | [LiveKit Server](/guide/rtc-livekit-server)                                                                               |
| **Host** | [Web](/web-sdk/host-mode) · [iOS](/ios-sdk/host-mode) · [Android](/android-sdk/host-mode)    | [Python](/server/python-sdk/python-host-mode) · [Golang](/server/go-sdk/go-host-mode) · [JS](/server/js-sdk/js-host-mode) |


# LiveKit Client
Source: https://docs.spatialreal.ai/guide/rtc-livekit-client

Real-time voice communication with avatars using LiveKit

```mermaid theme={null}
flowchart LR
    A["🎤 Microphone"] --> B["AvatarPlayer"]
    B <-->|WebRTC| C["LiveKit Server"]
    B --> D["AvatarKit SDK"]
    D --> E["🖥️ Avatar Rendering"]
```

<Note>
  **Web Only:** RTC Mode is currently available for Web applications only.
</Note>

<Warning>
  **Do not call `initializeAudioContext()`** — it is not needed in RTC Mode. Avatar audio is delivered as a native WebRTC audio track by the LiveKit client SDK, not through the AvatarKit SDK's internal audio player. The `AvatarPlayer` adapter only feeds **animation data** to the SDK for rendering; audio playback is handled entirely by LiveKit's WebRTC stack.
</Warning>

## Installation

<Warning>
  **Critical:** We are only compatable with `livekit-client` under version `2.17`, since `2.17` introduced single PC by default. We are working on adding support for `2.17`+ in a future release, but for now please ensure you install version `2.16.1` to avoid compatibility issues.
</Warning>

<Tabs>
  <Tab title="pnpm">
    ```bash theme={null}
    pnpm add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
    ```
  </Tab>

  <Tab title="npm">
    ```bash theme={null}
    npm install @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
    ```
  </Tab>

  <Tab title="yarn">
    ```bash theme={null}
    yarn add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
    ```
  </Tab>
</Tabs>

You also need to configure your build tool for WASM files — see [Build Tool Configuration](/guide/sdk-mode-web#build-tool-configuration).

## Authentication

| Credential        | How to Obtain                                                 | Notes                             |
| ----------------- | ------------------------------------------------------------- | --------------------------------- |
| **App ID**        | [SpatialReal Studio](https://app.spatialreal.ai) → Create App | For SDK initialization            |
| **Session Token** | Your backend → AvatarKit Server                               | For avatar loading (max 24 hours) |
| **LiveKit Token** | Your backend → LiveKit Server                                 | For RTC room connection           |

## Quick Start

For a complete client + server implementation, see [AvatarKit Voice Agent Demo](https://github.com/spatialwalk/avatarkit-voice-agent-demo/tree/main/livekit-cascade-voice-agent).

<Steps>
  <Step title="Initialize SDK in Host Mode">
    ```typescript theme={null}
    import { AvatarSDK, AvatarManager, AvatarView, DrivingServiceMode, Environment } from '@spatialwalk/avatarkit'

    await AvatarSDK.initialize('your-app-id', {
      environment: Environment.intl,
      drivingServiceMode: DrivingServiceMode.host,  // MUST be host for RTC
    })

    AvatarSDK.setSessionToken('your-session-token')
    ```
  </Step>

  <Step title="Load Avatar & Create View">
    ```typescript theme={null}
    const avatar = await AvatarManager.shared.load('avatar-id')
    const container = document.getElementById('avatar-container')!
    const avatarView = new AvatarView(avatar, container)
    ```
  </Step>

  <Step title="Create Player with LiveKit Provider">
    ```typescript theme={null}
    import { AvatarPlayer, LiveKitProvider } from '@spatialwalk/avatarkit-rtc'

    const provider = new LiveKitProvider()
    const player = new AvatarPlayer(provider, avatarView, {
      logLevel: 'warning',
    })
    ```
  </Step>

  <Step title="Connect to LiveKit Server">
    ```typescript theme={null}
    await player.connect({
      url: 'wss://your-livekit-server.com',
      token: 'your-livekit-token',
      roomName: 'room-name',
    })
    ```
  </Step>

  <Step title="Start Voice Interaction">
    ```typescript theme={null}
    // Start microphone publishing
    await player.startPublishing()

    // Stop microphone
    await player.stopPublishing()

    // Disconnect when done
    await player.disconnect()
    ```
  </Step>
</Steps>

## AvatarPlayer API

### Constructor

```typescript theme={null}
new AvatarPlayer(provider: LiveKitProvider, avatarView: AvatarView, options?: AvatarPlayerOptions)
```

### AvatarPlayerOptions

```typescript theme={null}
interface AvatarPlayerOptions {
  /** Start speaking transition frames, default 5 (~200ms at 25fps) */
  transitionStartFrameCount?: number

  /** End speaking transition frames, default 40 (~1600ms at 25fps) */
  transitionEndFrameCount?: number

  /** Log level: 'info' | 'warning' | 'error' | 'none', default 'warning' */
  logLevel?: LogLevel
}
```

### Connection

```typescript theme={null}
// Connect to LiveKit server
await player.connect(config: LiveKitConnectionConfig)

// Disconnect and clean up
await player.disconnect()

// Reconnect using last config (useful after stalls)
await player.reconnect()

// Check connection status
player.isConnected  // boolean
player.getConnectionState()  // ConnectionState
```

#### LiveKitConnectionConfig

```typescript theme={null}
interface LiveKitConnectionConfig {
  url: string       // LiveKit server URL (wss://...)
  token: string     // Auth token from your backend
  roomName: string  // Room name
}
```

### Microphone Control

```typescript theme={null}
// Start microphone (requests permission automatically)
await player.startPublishing()

// Stop microphone
await player.stopPublishing()
```

### Custom Audio Publishing

For non-microphone audio sources like audio elements or Web Audio API.

```typescript theme={null}
// Publish a custom audio track
await player.publishAudio(track: MediaStreamTrack)

// Stop custom audio
await player.unpublishAudio()
```

| Audio Source       | How to Obtain Track                                                      |
| ------------------ | ------------------------------------------------------------------------ |
| `<audio>` element  | `audioElement.captureStream().getAudioTracks()[0]`                       |
| Screen share audio | `getDisplayMedia({ audio: true })`                                       |
| Web Audio API      | `audioContext.createMediaStreamDestination().stream.getAudioTracks()[0]` |

<Note>
  `unpublishAudio()` does **not** stop the track. You are responsible for calling `track.stop()` to release the resource.
</Note>

### Native Client Access

Access the underlying LiveKit `Room` instance for advanced features.

```typescript theme={null}
import { LiveKitRoom } from '@spatialwalk/avatarkit-rtc'

// Via provider (recommended — full type safety)
const room = provider.getNativeClient()  // LiveKitRoom | null

// Via player (requires type assertion)
const room = player.getNativeClient() as LiveKitRoom | null

console.log('Remote participants:', room?.remoteParticipants.size)
```

### ConnectionState

```typescript theme={null}
type ConnectionState = 'disconnected' | 'connecting' | 'connected' | 'reconnecting' | 'failed'
```

## Events

```typescript theme={null}
player.on(event: string, handler: Function)
player.off(event: string, handler: Function)
```

| Event                        | Callback                   | Description                            |
| ---------------------------- | -------------------------- | -------------------------------------- |
| `'connected'`                | `()`                       | Connected to RTC server                |
| `'disconnected'`             | `()`                       | Disconnected from RTC server           |
| `'error'`                    | `(error: Error)`           | An error occurred                      |
| `'connection-state-changed'` | `(state: ConnectionState)` | Connection state changed               |
| `'stalled'`                  | `()`                       | Data stream stalled (no frames for 3s) |

<Note>
  When a `stalled` event fires, the avatar automatically transitions to idle animation. Consider calling `player.reconnect()` to recover.
</Note>

**Stall recovery example:**

```typescript theme={null}
player.on('stalled', async () => {
  console.log('Stream stalled, reconnecting...')
  try {
    await player.reconnect()
  } catch (error) {
    console.error('Reconnection failed:', error)
  }
})
```

## Complete Example

```typescript theme={null}
import { AvatarPlayer, LiveKitProvider } from '@spatialwalk/avatarkit-rtc'
import { AvatarSDK, AvatarView, AvatarManager, DrivingServiceMode, Environment } from '@spatialwalk/avatarkit'

async function init() {
  // Initialize SDK in host mode
  await AvatarSDK.initialize('your-app-id', {
    environment: Environment.intl,
    drivingServiceMode: DrivingServiceMode.host,
  })
  AvatarSDK.setSessionToken('your-session-token')

  // Load avatar and create view
  const avatar = await AvatarManager.shared.load('character-id')
  const container = document.getElementById('avatar-container')!
  const avatarView = new AvatarView(avatar, container)

  // Create player
  const provider = new LiveKitProvider()
  const player = new AvatarPlayer(provider, avatarView, {
    logLevel: 'info',
    transitionStartFrameCount: 5,
    transitionEndFrameCount: 40,
  })

  // Listen to events
  player.on('connected', () => console.log('Connected!'))
  player.on('disconnected', () => console.log('Disconnected!'))
  player.on('error', (err) => console.error('Error:', err))
  player.on('stalled', async () => {
    console.log('Stream stalled, reconnecting...')
    await player.reconnect()
  })

  // Connect to LiveKit server
  await player.connect({
    url: 'wss://your-livekit-server.com',
    token: 'your-livekit-token',
    roomName: 'my-room',
  })

  // Start microphone
  await player.startPublishing()
}
```

## Browser Compatibility

| Browser | Minimum Version | Notes                           |
| ------- | --------------- | ------------------------------- |
| Chrome  | 94+             | VP8 + RTCRtpScriptTransform     |
| Firefox | 117+            | RTCRtpScriptTransform required  |
| Safari  | 15.4+           | RTCRtpScriptTransform supported |
| Edge    | 94+             | Same as Chrome                  |


# Serverside Setup With LiveKit
Source: https://docs.spatialreal.ai/guide/rtc-livekit-server

Server-side setup for LiveKit + SpatialReal avatar

This guide covers how to send avatar audio and animation to a LiveKit room from your backend. Choose the path that matches how you build your voice agent.

***

## 1. Using the LiveKit Agents Framework

If you already use [LiveKit Agents](https://docs.livekit.io/agents/) to build your voice agent, the simplest option is [livekit-plugins-spatialreal](https://github.com/spatialwalk/livekit-plugins-spatialreal). The plugin hooks into your agent pipeline, sends TTS audio to SpatialReal, and publishes the lip-synced avatar stream into your LiveKit room—so you don’t manage the Server SDK or egress config yourself.

For a working end-to-end reference, see [AvatarKit Voice Agent Demo](https://github.com/spatialwalk/avatarkit-voice-agent-demo/tree/main/livekit-cascade-voice-agent).

### How it works

1. Your agent runs as usual (VAD, STT, LLM, TTS or Realtime), your current AgentSession setup.
2. The plugin intercepts TTS audio from the agent and sends it to SpatialReal.
3. SpatialReal generates the avatar stream and publishes it to the same LiveKit room.
4. Your client join the room and use the SpatialReal RTC client to render the avatar.

Interruption and conversation state are handled inside the plugin.

### Python: livekit-plugins-spatialreal

**Install**

```bash theme={null}
pip install livekit-plugins-spatialreal
```

**Configure**

| Variable                       | Required | Description               |
| ------------------------------ | -------- | ------------------------- |
| `SPATIALREAL_API_KEY`          | Yes      | Your SpatialReal API key  |
| `SPATIALREAL_APP_ID`           | Yes      | Your SpatialReal app ID   |
| `SPATIALREAL_AVATAR_ID`        | Yes      | Avatar to use             |
| `SPATIALREAL_CONSOLE_ENDPOINT` | No       | Override console endpoint |
| `SPATIALREAL_INGRESS_ENDPOINT` | No       | Override ingress endpoint |
| `LIVEKIT_URL`                  | Yes      | Your LiveKit server URL   |
| `LIVEKIT_API_KEY`              | Yes      | LiveKit API key           |
| `LIVEKIT_API_SECRET`           | Yes      | LiveKit API secret        |

**Use in your agent**

Create an `AvatarSession` and start it with your agent session and room. The plugin will attach to the pipeline and publish the avatar to the room.

```python theme={null}
from livekit.agents import Agent, AgentSession, JobContext, cli, WorkerOptions
from livekit.plugins import spatialreal

async def entrypoint(ctx: JobContext):
    await ctx.connect()

    session = AgentSession(vad=vad, stt=stt, llm=llm, tts=tts)
    avatar = spatialreal.AvatarSession()
    await avatar.start(session, room=ctx.room)

    await session.start(agent=YourAgent(), room=ctx.room)

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
```

Full API and options: **[livekit-plugins-spatialreal](https://github.com/spatialwalk/livekit-plugins-spatialreal)** (PyPI: [livekit-plugins-spatialreal](https://pypi.org/project/livekit-plugins-spatialreal/)).

### JavaScript plugin (coming soon)

JavaScript/TypeScript plugin for the LiveKit Agents framework will be available soon.

***

## 2. Using the Server SDK with LiveKit Egress

If you are **not** using LiveKit Agents (e.g. custom voice agent server), you can still send avatar output to a LiveKit room by using the **SpatialReal AvatarKit Server SDK** with **LiveKit egress** enabled.

You can also use the [AvatarKit Voice Agent Demo](https://github.com/spatialwalk/avatarkit-voice-agent-demo/tree/main/livekit-cascade-voice-agent) as a concrete reference for wiring a LiveKit-based voice agent.

In this setup:

* Your server sends audio to the avatar service and passes **LiveKit egress config** (room URL, credentials, room name, publisher id).
* The avatar service streams **audio + animation** directly into the LiveKit room (no need for your server to relay that data).
* Clients join the room and use [@spatialwalk/avatarkit-rtc](/guide/rtc-livekit-client) to render the avatar.

You keep full control over when and how you send audio (and optional interrupt signals; see below), while transport and sync are handled by the service and the RTC client.

### Configuration

| Field          | Description                                                |
| -------------- | ---------------------------------------------------------- |
| `url`          | LiveKit server URL (e.g., `wss://your-livekit-server.com`) |
| `api_key`      | LiveKit API key                                            |
| `api_secret`   | LiveKit API secret                                         |
| `room_name`    | LiveKit room name to publish to                            |
| `publisher_id` | Publisher identity in the room                             |

### Golang Example

```go theme={null}
session := avatarsdkgo.NewAvatarSession(
	avatarsdkgo.WithAPIKey("your-api-key"),
	avatarsdkgo.WithAppID("your-app-id"),
	avatarsdkgo.WithAvatarID("your-avatar-id"),
	avatarsdkgo.WithConsoleEndpointURL("https://console.us-west.spatialwalk.cloud/v1/console"),
	avatarsdkgo.WithIngressEndpointURL("wss://api.us-west.spatialwalk.cloud/v2/driveningress"),
	avatarsdkgo.WithExpireAt(time.Now().Add(5 * time.Minute)),
	avatarsdkgo.WithLiveKitEgress(&avatarsdkgo.LiveKitEgressConfig{
		URL:         "wss://your-livekit-server.com",
		APIKey:      "livekit-api-key",
		APISecret:   "livekit-api-secret",
		RoomName:    "your-room-name",
		PublisherID: "avatar-publisher",
	}),
	avatarsdkgo.WithOnError(func(err error) {
		// handle error
	}),
	avatarsdkgo.WithOnClose(func() {
		// handle close
	}),
)
```

### Python Example

```python theme={null}
from avatarkit import new_avatar_session, LiveKitEgressConfig

session = new_avatar_session(
    api_key="your-api-key",
    app_id="your-app-id",
    avatar_id="your-avatar-id",
    expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
    console_endpoint_url="https://console.us-west.spatialwalk.cloud/v1/console",
    ingress_endpoint_url="wss://api.us-west.spatialwalk.cloud/v2/driveningress",
    livekit_egress=LiveKitEgressConfig(
        url="wss://your-livekit-server.com",
        api_key="livekit-api-key",
        api_secret="livekit-api-secret",
        room_name="your-room-name",
        publisher_id="avatar-publisher",
    ),
    on_error=lambda err: print(f"Error: {err}"),
    on_close=lambda: print("Session closed"),
)
```

After the session is created with LiveKit egress, send audio as in normal Server SDK usage; the service will publish to the configured room automatically.

### Important Notes

When LiveKit egress is enabled:

* The `TransportFrames` / `transport_frames` callback will **not** be invoked
* Audio and animation data are published directly to the specified LiveKit room
* Your client must use the **@spatialwalk/avatarkit-rtc** package to render the avatar (standard video players won't work). See [LiveKit Client Guide](/guide/rtc-livekit-client) for client setup.

### Interrupt

You can interrupt the avatar (e.g. when the user asks a new question). The interrupt uses the most recent request ID, even after `end=true` was sent.

**Golang**

```go theme={null}
// Send audio
requestID, err := session.SendAudio(audioData, true)

// Later, interrupt if needed
interruptedID, err := session.Interrupt()
```

**Python**

```python theme={null}
# Send audio
request_id = await session.send_audio(audio_data, end=True)

# Later, interrupt if needed
interrupted_id = await session.interrupt()
```

***

## Summary

| Approach                        | Best for                                                           | Where to go                                                                                                      |
| ------------------------------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------- |
| **LiveKit Agents + Plugin**     | Python (or future JS) agents using the LiveKit Agents framework    | [livekit-plugins-spatialreal](https://github.com/spatialwalk/livekit-plugins-spatialreal) (this page, section 1) |
| **Server SDK + LiveKit egress** | Custom backends, any language with a Server SDK (Go, Python, etc.) | This page, section 2                                                                                             |

For client-side setup (joining the room and rendering the avatar), see [LiveKit client guide](/guide/rtc-livekit-client).


# RTC Mode & Framework Plugin
Source: https://docs.spatialreal.ai/guide/rtc-mode

Use LiveKit as the transport layer for SpatialReal avatars

## What is RTC Mode?

RTC Mode enables **real-time voice communication** with avatars through LiveKit. It builds on top of [Host Mode](/guide/host-mode) — the `@spatialwalk/avatarkit-rtc` adapter package handles the RTC connection and feeds audio/animation data to the SDK automatically.

```mermaid theme={null}
flowchart LR
    A["🎤 Microphone"] --> B["AvatarPlayer<br/>(RTC Adapter)"]
    B <-->|WebRTC| C["RTC Server<br/>(LiveKit)"]
    B --> D["AvatarKit SDK"]
    D --> E["🖥️ Avatar Rendering"]
```

<Note>
  **Web Only:** RTC Mode is currently available for Web applications only. iOS and Android support is planned for a future release. In the meantime, native mobile platforms can use [Host Mode](/guide/host-mode) with your own RTC implementation.
</Note>

<Card title="AvatarKit Voice Agent Demo" icon="github" href="https://github.com/spatialwalk/avatarkit-voice-agent-demo">
  A full reference repository with implementation details, including different frontend UI options and multiple backend agent patterns.
</Card>

## When to Use

* **Real-time voice conversation** — users talk to an avatar via microphone
* **Low-latency interaction** — WebRTC provides sub-second latency
* **Server-side AI** — your RTC server processes audio and generates responses

## Packages Required

| Package                      | Purpose              | Required |
| ---------------------------- | -------------------- | -------- |
| `@spatialwalk/avatarkit`     | Avatar rendering SDK | Yes      |
| `@spatialwalk/avatarkit-rtc` | RTC adapter          | Yes      |
| `livekit-client@2.16.1`      | LiveKit RTC SDK      | Yes      |

<Warning>
  **Critical:** We are only compatable with `livekit-client` under version `2.17`, since `2.17` introduced single PC by default. We are working on adding support for `2.17`+ in a future release, but for now please ensure you install version `2.16.1` to avoid compatibility issues.
</Warning>

## LiveKit Browser Compatibility

| Browser     | Minimum Version |
| ----------- | --------------- |
| **Chrome**  | 94+             |
| **Firefox** | 117+            |
| **Safari**  | 15.4+           |
| **Edge**    | 94+             |

## How It Works

RTC Mode uses the SDK in **Host Mode** internally. The `AvatarPlayer` acts as a bridge:

1. Initializes the avatar SDK with `DrivingServiceMode.host`
2. Connects to the RTC server via the chosen provider
3. Publishes your microphone audio to the RTC server
4. Receives animation and audio data from the RTC server
5. Feeds **animation data** into the avatar SDK for rendering; **audio is played through the native WebRTC audio track** by the RTC provider

You don't need to call `yieldAudioData()` or `yieldFramesData()` manually — the adapter handles this.

### What RTC Mode Does NOT Use

Although RTC Mode builds on Host Mode internally, the following SDK/Host Mode features are **not used** in RTC Mode:

| Feature                                  | Used in SDK/Host Mode                        | Used in RTC Mode                             |
| ---------------------------------------- | -------------------------------------------- | -------------------------------------------- |
| `initializeAudioContext()`               | Yes — required for Web Audio API playback    | **No** — audio is played via WebRTC tracks   |
| Internal audio player                    | Yes — SDK decodes and plays audio internally | **No** — RTC provider handles audio playback |
| `yieldAudioData()` / `yieldFramesData()` | Yes — you call these manually                | **No** — the adapter calls them internally   |
| `start()` / `send()` / `close()`         | Yes (SDK Mode only)                          | **No**                                       |

<Warning>
  **Audio path difference:** In SDK Mode and Host Mode, the SDK plays avatar audio through its internal audio player (Web Audio API). In RTC Mode, avatar audio arrives as a **native WebRTC audio track** and is played by the browser's WebRTC stack directly — the SDK's internal audio player is not involved. This distinction matters for audio processing (e.g., echo cancellation, noise suppression).
</Warning>

## Server-Side Setup

Your backend is responsible for sending audio to the avatar service and having the resulting avatar stream published to your RTC room. Two approaches are available:

| Platform    | Framework plugin                                                                                                                               | Server SDK + egress                                                                                           |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| **LiveKit** | [LiveKit Agents plugin](https://github.com/spatialwalk/livekit-plugins-spatialreal) — hooks into your agent pipeline and publishes to the room | [LiveKit Server](/guide/rtc-livekit-server) (section 2) — use AvatarKit Server SDK with LiveKit egress config |

* **Framework plugin**: Best if you already use LiveKit Agents. The plugin handles audio → SpatialReal → RTC publish for you.
* **Server SDK + egress**: Use the [Golang](/server/go-sdk/go-sdk) or [Python](/server/python-sdk/python-sdk) Server SDK, create a session with LiveKit egress config, and send audio. The avatar service publishes audio and animation directly to the RTC room; your server does not relay that data.

Client-side setup is the same either way: the client joins the room and uses `@spatialwalk/avatarkit-rtc` to render the avatar. See the client guides below.

## Get Started

**Client (browser)**

<CardGroup>
  <Card title="LiveKit Client" icon="bolt" href="/guide/rtc-livekit-client">
    Connect with LiveKit RTC
  </Card>
</CardGroup>

**Server (backend)**

<CardGroup>
  <Card title="LiveKit Server" icon="bolt" href="/guide/rtc-livekit-server">
    Plugin or Server SDK with LiveKit egress
  </Card>
</CardGroup>


# SDK Mode
Source: https://docs.spatialreal.ai/guide/sdk-mode

Server-driven avatar animation via WebSocket

## What is SDK Mode?

SDK Mode is the **default** integration mode. The SDK handles all network communication with AvatarKit servers automatically — you just send audio data, and the SDK takes care of fetching animation data, synchronizing playback, and rendering.

```mermaid theme={null}
flowchart LR
    A["🎤 Audio Input<br/>(PCM16 mono)"] -->|send| B["AvatarKit SDK"]
    B -->|WebSocket| C["AvatarKit Server"]
    C -->|Animation Data| B
    B --> D["🖥️ Avatar Rendering"]
```

## When to Use

* **Real-time audio input** — microphone or audio file playback driving an avatar
* **Simplest integration** — minimal code, SDK handles networking
* **Server-side processing** — animation data is generated by AvatarKit backend

## Requirements

| Requirement       | Description                                                    |
| ----------------- | -------------------------------------------------------------- |
| **App ID**        | Obtained from [SpatialReal Studio](https://app.spatialreal.ai) |
| **Session Token** | Obtained from your backend server (max 24 hours validity)      |
| **Audio Format**  | PCM16, mono, configurable sample rate (default 16000 Hz)       |

<Note>
  **Authentication Flow:**

  ```
  Your Client → Your Backend → AvatarKit Server → Session Token (24 hours max)
  ```

  The Session Token must be set before calling `start()`. See the platform-specific guides below for details.
</Note>

## Platform Comparison

| Feature          | Web                                        | iOS                           | Android                        |
| ---------------- | ------------------------------------------ | ----------------------------- | ------------------------------ |
| **Package**      | `@spatialwalk/avatarkit`                   | `AvatarKit.xcframework` / SPM | `ai.spatialwalk:avatarkit`     |
| **Rendering**    | WebGL / WebGPU                             | Metal                         | Vulkan                         |
| **UI Framework** | DOM Canvas                                 | UIKit + SwiftUI wrapper       | Android View + Compose wrapper |
| **Audio Init**   | `initializeAudioContext()` in user gesture | Automatic                     | Automatic                      |
| **Build Config** | Vite plugin / Next.js wrapper required     | Xcode linker flags            | Gradle dependency              |

## Key Concepts

### Fallback Mechanism

If the WebSocket connection fails within 15 seconds, the SDK automatically enters **audio-only fallback mode** — audio continues to play normally without animation. This ensures uninterrupted audio playback even when the server is unreachable.

### ConversationId

Every `send()` call returns a `conversationId` that identifies the current conversation round. When `end: true` is passed, the round is completed. Sending new audio after that starts a new round and interrupts any ongoing playback.

## Get Started

<CardGroup>
  <Card title="Web" icon="globe" href="/guide/sdk-mode-web">
    JavaScript / TypeScript
  </Card>

  <Card title="iOS Demo" icon="apple" href="https://github.com/spatialwalk/avatar-kit-ios-demo">
    Use the GitHub demo as the iOS guide
  </Card>

  <Card title="Android Demo" icon="android" href="https://github.com/spatialwalk/avatar-kit-android-demo">
    Use the GitHub demo as the Android guide
  </Card>
</CardGroup>


# SDK Mode
Source: https://docs.spatialreal.ai/guide/sdk-mode-ios&android

Server-driven avatar animation via WebSocket for iOS and Android applications

Use the official demo repositories as the primary integration guides.

<CardGroup>
  <Card title="iOS Demo" icon="apple" href="https://github.com/spatialwalk/avatar-kit-ios-demo">
    Swift / SwiftUI reference implementation.
  </Card>

  <Card title="Android Demo" icon="android" href="https://github.com/spatialwalk/avatar-kit-android-demo">
    Kotlin / Compose reference implementation.
  </Card>
</CardGroup>

Need full API details? Use the SDK references below.

<CardGroup>
  <Card title="iOS SDK Reference" icon="book" href="/sdk-reference/ios-sdk/api-reference">
    Complete iOS API documentation.
  </Card>

  <Card title="Android SDK Reference" icon="book" href="/sdk-reference/android-sdk/api-reference">
    Complete Android API documentation.
  </Card>
</CardGroup>


# SDK Mode
Source: https://docs.spatialreal.ai/guide/sdk-mode-web

Server-driven avatar animation via WebSocket for Web applications

```mermaid theme={null}
flowchart LR
    A["🎤 Audio Input"] -->|send| B["AvatarKit SDK<br/>(Web)"]
    B -->|WebSocket| C["AvatarKit Server"]
    C -->|Animation Data| B
    B --> D["🖥️ WebGL / WebGPU"]
```

<CardGroup>
  <Card title="Web Demo Repository" icon="github" href="https://github.com/spatialwalk/avatar-kit-web-demo">
    Complete working examples for Vanilla JS, Vue, and React — the fastest way to get started.
  </Card>
</CardGroup>

## Installation

<Tabs>
  <Tab title="pnpm">
    ```bash theme={null}
    pnpm add @spatialwalk/avatarkit
    ```
  </Tab>

  <Tab title="npm">
    ```bash theme={null}
    npm install @spatialwalk/avatarkit
    ```
  </Tab>

  <Tab title="yarn">
    ```bash theme={null}
    yarn add @spatialwalk/avatarkit
    ```
  </Tab>
</Tabs>

## Build Tool Configuration

<Warning>
  **Required:** The SDK uses WASM files that need special build configuration. You **must** configure your build tool before using the SDK.
</Warning>

<Tabs>
  <Tab title="Vite">
    Add the official plugin to `vite.config.ts`:

    ```typescript title="vite.config.ts" theme={null}
    import { defineConfig } from 'vite'
    import { avatarkitVitePlugin } from '@spatialwalk/avatarkit/vite'

    export default defineConfig({
      plugins: [
        avatarkitVitePlugin(),
      ],
    })
    ```

    <Accordion title="Plugin Features">
      The plugin automatically handles:

      * **Development Server**: Sets correct MIME type for WASM files
      * **Build Time**: Copies WASM files to `dist/assets/`
      * **Cloudflare Pages**: Generates `_headers` file
      * **Vite Configuration**: Configures `optimizeDeps`, `assetsInclude`, etc.
    </Accordion>

    <Accordion title="Manual Configuration (Without Plugin)">
      ```typescript title="vite.config.ts" theme={null}
      import { defineConfig } from 'vite'

      export default defineConfig({
        optimizeDeps: {
          exclude: ['@spatialwalk/avatarkit'],
        },
        assetsInclude: ['**/*.wasm'],
        build: {
          assetsInlineLimit: 0,
          rollupOptions: {
            output: {
              assetFileNames: (assetInfo) => {
                if (assetInfo.name?.endsWith('.wasm')) {
                  return 'assets/[name][extname]'
                }
                return 'assets/[name]-[hash][extname]'
              },
            },
          },
        },
        configureServer(server) {
          server.middlewares.use((req, res, next) => {
            if (req.url?.endsWith('.wasm')) {
              res.setHeader('Content-Type', 'application/wasm')
            }
            next()
          })
        },
      })
      ```
    </Accordion>
  </Tab>

  <Tab title="Next.js">
    Wrap your config in `next.config.mjs`:

    ```javascript title="next.config.mjs" theme={null}
    import { withAvatarkit } from '@spatialwalk/avatarkit/next'

    export default withAvatarkit({
      // ...your existing Next.js config
    })
    ```

    <Accordion title="Wrapper Features">
      The wrapper automatically handles:

      * **Emscripten Fix**: Patches `scriptDirectory` so the WASM glue file correctly resolves assets at `/_next/static/chunks/`
      * **WASM Copying**: Copies `.wasm` files into `static/chunks/` via a custom webpack plugin (client build only)
      * **Content-Type Headers**: Adds `application/wasm` response header for `/_next/static/chunks/*.wasm`
      * **Config Chaining**: Preserves your existing `webpack` and `headers` configurations
    </Accordion>

    <Accordion title="Multiple Config Wrappers">
      If you have multiple config wrappers, `withAvatarkit` must wrap your entire config:

      ```javascript title="next.config.mjs" theme={null}
      import { withAvatarkit } from '@spatialwalk/avatarkit/next'
      import withOtherPlugin from 'other-plugin'

      export default withAvatarkit(withOtherPlugin({
        // ...your config
      }))
      ```
    </Accordion>
  </Tab>
</Tabs>

## Authentication

SDK Mode requires an **App ID** and a **Session Token**.

| Credential        | How to Obtain                                                 | Notes                  |
| ----------------- | ------------------------------------------------------------- | ---------------------- |
| **App ID**        | [SpatialReal Studio](https://app.spatialreal.ai) → Create App | Required for all modes |
| **Session Token** | Your backend server requests it from AvatarKit Server         | Max 24 hours validity  |

<Note>
  **Authentication Flow:**

  ```
  Your Client → Your Backend → AvatarKit Server → Session Token (24 hours max)
  ```

  The Session Token must be set before calling `start()`. Never expose your token generation logic in client-side code.
</Note>

## Quick Start

<Steps>
  <Step title="Initialize SDK">
    ```typescript theme={null}
    import {
      AvatarSDK,
      Environment,
      DrivingServiceMode,
    } from '@spatialwalk/avatarkit'

    await AvatarSDK.initialize('your-app-id', {
      environment: Environment.intl,  // or Environment.cn
      drivingServiceMode: DrivingServiceMode.sdk,  // Default
    })

    // Set session token (required before start())
    AvatarSDK.setSessionToken('your-session-token')
    ```
  </Step>

  <Step title="Load Avatar">
    ```typescript theme={null}
    import { AvatarManager } from '@spatialwalk/avatarkit'

    const avatar = await AvatarManager.shared.load('avatar-id', (progress) => {
      console.log(`Loading: ${progress.progress}%`)
    })
    ```
  </Step>

  <Step title="Create View">
    ```typescript theme={null}
    import { AvatarView } from '@spatialwalk/avatarkit'

    // Container MUST have non-zero width and height
    const container = document.getElementById('avatar-container')!
    const avatarView = new AvatarView(avatar, container)
    ```
  </Step>

  <Step title="Initialize Audio Context">
    <Warning>
      **Critical:** `initializeAudioContext()` **must** be called inside a user gesture handler (e.g., `click`, `touchstart`). This is a browser security requirement — calling it outside a user gesture will fail silently.
    </Warning>

    ```typescript theme={null}
    button.addEventListener('click', async () => {
      await avatarView.controller.initializeAudioContext()
    })
    ```
  </Step>

  <Step title="Connect and Send Audio">
    ```typescript theme={null}
    // Start WebSocket connection to AvatarKit server
    await avatarView.controller.start()

    // Send audio data (PCM16, mono, matching configured sample rate)
    const audioData: ArrayBuffer = /* your PCM16 audio data */
    avatarView.controller.send(audioData, false)  // Continue sending

    // Mark end of conversation round
    avatarView.controller.send(lastChunk, true)
    ```
  </Step>

  <Step title="Cleanup">
    ```typescript theme={null}
    avatarView.controller.close()  // Close WebSocket connection
    avatarView.dispose()           // Release all resources
    ```
  </Step>
</Steps>

## Core API

### AvatarSDK

SDK initialization and global configuration.

```typescript theme={null}
// Initialize
await AvatarSDK.initialize(appId: string, configuration: Configuration)

// Properties (read-only)
AvatarSDK.isInitialized   // boolean
AvatarSDK.appId           // string
AvatarSDK.configuration   // Configuration
AvatarSDK.version         // string
AvatarSDK.sessionToken    // string

// Methods
AvatarSDK.setSessionToken(token: string)  // Set auth token
AvatarSDK.setUserId(userId: string)       // Set user ID (for telemetry)
AvatarSDK.cleanup()                       // Release all SDK resources
```

<Note>
  `setSessionToken()` can be called before or after `initialize()`. If called before, the token is applied automatically during initialization.
</Note>

### AvatarManager

Avatar resource loading and caching. Access via the singleton `AvatarManager.shared`.

```typescript theme={null}
const manager = AvatarManager.shared

// Load avatar (downloads and caches)
const avatar = await manager.load(
  id: string,
  onProgress?: (progress: LoadProgressInfo) => void
)

// Clear all cached resources
manager.clearAll()
```

| Parameter    | Type                 | Description                                           |
| ------------ | -------------------- | ----------------------------------------------------- |
| `id`         | `string`             | Avatar character ID                                   |
| `onProgress` | `(progress) => void` | Progress callback with `{ progress: number }` (0–100) |

### AvatarView

3D rendering view. Automatically creates a Canvas element and an `AvatarController`.

```typescript theme={null}
// Create view — Canvas is added to container automatically
const avatarView = new AvatarView(avatar: Avatar, container: HTMLElement)
```

| Property / Method  | Type               | Description                       |
| ------------------ | ------------------ | --------------------------------- |
| `controller`       | `AvatarController` | Playback controller (read-only)   |
| `transform`        | `{ x, y, scale }`  | Avatar position and scale         |
| `onFirstRendering` | `() => void`       | Callback when first frame renders |
| `dispose()`        | `void`             | Release all resources             |

**Transform coordinates:**

| Field   | Range   | Description                                          |
| ------- | ------- | ---------------------------------------------------- |
| `x`     | -1 to 1 | Horizontal offset (-1 = left, 0 = center, 1 = right) |
| `y`     | -1 to 1 | Vertical offset (-1 = bottom, 0 = center, 1 = top)   |
| `scale` | > 0     | Scale factor (1.0 = original size)                   |

<Warning>
  **Container requirement:** The container element **must** have non-zero `width` and `height`. The Canvas fills the container and auto-resizes via `ResizeObserver`.
</Warning>

### AvatarController — SDK Mode Methods

These methods are only available when `drivingServiceMode` is `DrivingServiceMode.sdk`.

```typescript theme={null}
// Initialize audio (MUST be in user gesture handler)
await controller.initializeAudioContext()

// Connect to AvatarKit server
await controller.start()

// Send audio data — returns conversationId
const conversationId = controller.send(
  audioData: ArrayBuffer,  // PCM16, mono
  end: boolean             // true = end of conversation round
)

// Close connection
controller.close()
```

**`send()` behavior:**

* `end: false` — continues the current conversation round
* `end: true` — marks the end of the current round; sending new audio after this starts a new round and interrupts any ongoing playback

### AvatarController — Common Methods

Available in both SDK Mode and Host Mode.

```typescript theme={null}
// Playback control
controller.pause()       // Pause audio + animation
controller.resume()      // Resume playback
controller.interrupt()   // Stop current playback, clear data

// Data management
controller.clear()       // Clear all data and resources

// Conversation
controller.getCurrentConversationId()  // string | null

// Volume (affects avatar audio only, not system volume)
controller.setVolume(volume: number)   // 0.0 to 1.0
controller.getVolume(): number         // Current volume
```

### AvatarController — Event Callbacks

```typescript theme={null}
// Connection state changes (SDK Mode only)
controller.onConnectionState = (state: ConnectionState) => {
  console.log('Connection:', state)
}

// Conversation state changes
controller.onConversationState = (state: ConversationState) => {
  console.log('Conversation:', state)
}

// Error handler
controller.onError = (error: Error) => {
  console.error('Error:', error)
}
```

## Audio Format

The SDK requires audio in **mono PCM16** format:

| Property        | Value                                                                            |
| --------------- | -------------------------------------------------------------------------------- |
| **Format**      | PCM16 (16-bit signed integer, little-endian)                                     |
| **Channels**    | Mono (1 channel)                                                                 |
| **Sample Rate** | Configurable: 8000, 16000, 22050, 24000, 32000, 44100, 48000 Hz (default: 16000) |
| **Data Type**   | `ArrayBuffer` or `Uint8Array`                                                    |

**Data size:** 1 second at 16 kHz = 16,000 samples × 2 bytes = 32,000 bytes.

<Accordion title="Converting MP3 to PCM16">
  ```typescript theme={null}
  async function mp3ToPcm16(mp3File: File, targetSampleRate: number): Promise<ArrayBuffer> {
    const arrayBuffer = await mp3File.arrayBuffer()
    const audioContext = new AudioContext({ sampleRate: targetSampleRate })
    const audioBuffer = await audioContext.decodeAudioData(arrayBuffer.slice(0))

    const length = audioBuffer.length
    const channels = audioBuffer.numberOfChannels
    const pcm16Buffer = new ArrayBuffer(length * 2)
    const pcm16View = new DataView(pcm16Buffer)

    // Mix to mono if stereo
    const mono = channels === 1
      ? audioBuffer.getChannelData(0)
      : (() => {
          const mixed = new Float32Array(length)
          const left = audioBuffer.getChannelData(0)
          const right = audioBuffer.getChannelData(1)
          for (let i = 0; i < length; i++) mixed[i] = (left[i] + right[i]) / 2
          return mixed
        })()

    // Float32 → Int16
    for (let i = 0; i < length; i++) {
      const s = Math.max(-1, Math.min(1, mono[i]))
      pcm16View.setInt16(i * 2, s < 0 ? s * 0x8000 : s * 0x7FFF, true)
    }

    audioContext.close()
    return pcm16Buffer
  }
  ```
</Accordion>

## Configuration

### Configuration Interface

```typescript theme={null}
interface Configuration {
  environment: Environment
  drivingServiceMode?: DrivingServiceMode  // Default: DrivingServiceMode.sdk
  logLevel?: LogLevel                      // Default: LogLevel.off
  audioFormat?: AudioFormat                // Default: { channelCount: 1, sampleRate: 16000 }
}
```

### Environment

```typescript theme={null}
enum Environment {
  cn = 'cn',      // China region
  intl = 'intl',  // International region
}
```

### DrivingServiceMode

```typescript theme={null}
enum DrivingServiceMode {
  sdk = 'sdk',    // Server-driven (default)
  host = 'host',  // Client-driven
}
```

### AudioFormat

```typescript theme={null}
interface AudioFormat {
  readonly channelCount: 1   // Fixed to mono
  readonly sampleRate: number // 8000 | 16000 | 22050 | 24000 | 32000 | 44100 | 48000
}
```

### LogLevel

```typescript theme={null}
enum LogLevel {
  off = 'off',          // No logging (default)
  error = 'error',      // Errors only
  warning = 'warning',  // Errors + warnings
  all = 'all',          // All logs
}
```

## State Management

### ConnectionState

Reported via `onConnectionState` callback (SDK Mode only).

```typescript theme={null}
enum ConnectionState {
  disconnected = 'disconnected',
  connecting = 'connecting',
  connected = 'connected',
  failed = 'failed',
}
```

### ConversationState

Reported via `onConversationState` callback.

```typescript theme={null}
enum ConversationState {
  idle = 'idle',        // Breathing animation, waiting for input
  playing = 'playing',  // Active conversation playback
  pausing = 'pausing',  // Paused during playback
}
```

<Note>
  State transitions are notified immediately when the transition starts, not when the animation completes. For example, `playing` is reported as soon as the transition from `idle` begins.
</Note>

## Error Handling

### AvatarError

```typescript theme={null}
import { AvatarError } from '@spatialwalk/avatarkit'

try {
  await avatarView.controller.start()
} catch (error) {
  if (error instanceof AvatarError) {
    console.error('SDK error:', error.message, error.code)
  }
}
```

### Error Callback

```typescript theme={null}
avatarView.controller.onError = (error: Error) => {
  console.error('Controller error:', error)
}
```

## Lifecycle Management

### Avatar Switching

```typescript theme={null}
// 1. Dispose current view
currentAvatarView.dispose()

// 2. Load new avatar
const newAvatar = await AvatarManager.shared.load('new-avatar-id')

// 3. Create new view (reuse same container)
currentAvatarView = new AvatarView(newAvatar, container)

// 4. Reconnect
await currentAvatarView.controller.initializeAudioContext()
await currentAvatarView.controller.start()
```

### Resource Cleanup

`dispose()` automatically cleans up all resources:

* WebSocket connections
* Audio playback data and animation resources
* Canvas elements and render system
* Event listeners and callbacks

<Warning>
  Always call `dispose()` when the view is no longer needed. Failing to do so may cause memory leaks.
</Warning>

### Fallback Mechanism

If the WebSocket connection fails within 15 seconds, the SDK automatically enters **audio-only fallback mode** — audio continues playing without animation. This ensures uninterrupted audio playback when the server is unreachable.

* The fallback mode is interruptible like normal playback
* `onConnectionState` reports `failed` when the connection times out

## Browser Compatibility

| Browser        | Minimum Version | Rendering          |
| -------------- | --------------- | ------------------ |
| Chrome / Edge  | 90+             | WebGPU (preferred) |
| Firefox        | 90+             | WebGL              |
| Safari         | 14+             | WebGL              |
| iOS Safari     | 14+             | WebGL              |
| Android Chrome | 90+             | WebGL              |

## Common Issues

| Issue                       | Cause                                          | Solution                                                              |
| --------------------------- | ---------------------------------------------- | --------------------------------------------------------------------- |
| Audio not working           | `initializeAudioContext()` not in user gesture | Call it inside a `click` or `touchstart` handler                      |
| Avatar not rendering        | Container has zero dimensions                  | Set explicit `width` and `height` on the container                    |
| WASM MIME type error        | Build tool misconfiguration                    | Use Vite plugin or Next.js wrapper                                    |
| Session token invalid       | Token expired or not set                       | Refresh token from backend, call `setSessionToken()` before `start()` |
| WebSocket connection failed | Network or auth issue                          | Check network connectivity and token validity                         |


# Agent
Source: https://docs.spatialreal.ai/overview/agent

Rapidly integrate SpatialReal Avatar with Vue + LiveKit + end-to-end model

Using this quickstart to create a Agent demo with **SpatialReal SDK** and LiveKit Agents.
In a few steps, you will set up a backend that uses an end-to-end model and connects to a **SpatialReal** session. Then, you will create a frontend Vue app that renders the **SpatialReal avatar** and streams microphone audio to the agent in real time.

<Note>
  You can just copy each code block as-is and replace values in `.env`.
  Or just dump this page to any LLM you like and ask it to implement this demo for you.
</Note>

<Card title="Github Repository of this quickstart" icon="github" href="https://github.com/spatialwalk/avatarkit-voice-agent-demo/tree/main/spatialreal-agent-quickstart" />

## Prerequisites

* SpatialReal account (`SPATIALREAL_API_KEY`, `SPATIALREAL_APP_ID`, `SPATIALREAL_AVATAR_ID`)
* LiveKit Cloud project (`LIVEKIT_URL`, `LIVEKIT_API_KEY`, `LIVEKIT_API_SECRET`)
* End-to-end model key for Gemini Live (`GOOGLE_API_KEY`)
* Node.js 18+, Python 3.10+, `pnpm`, `uv`

<Steps>
  <Step title="Create workspace">
    Create one project folder, then scaffold a Vue frontend and a backend folder.
    You will use this split layout for all files in the next steps.

    ```bash theme={null}
    mkdir spatialreal-agent-quickstart
    cd spatialreal-agent-quickstart

    mkdir backend
    pnpm dlx create-vite@latest frontend --template vue-ts
    ```
  </Step>

  <Step title="Configure backend">
    Add these backend files exactly as shown.

    Create `backend/.env` and paste your LiveKit, Gemini Live, and SpatialReal credentials.
    Both backend processes read from this file at runtime.

    ```bash title="backend/.env" theme={null}
    LIVEKIT_URL=wss://your-project.livekit.cloud # https://cloud.livekit.io/
    LIVEKIT_API_KEY=your_api_key # LiveKit Cloud -> API Keys
    LIVEKIT_API_SECRET=your_api_secret # LiveKit Cloud -> API Keys

    GOOGLE_API_KEY=your_google_api_key # https://aistudio.google.com/apikey
    E2E_GOOGLE_MODEL=gemini-2.5-flash-native-audio-preview-12-2025
    E2E_GOOGLE_VOICE=Puck

    SPATIALREAL_API_KEY=your_key # https://app.spatialreal.ai/apps
    SPATIALREAL_APP_ID=your_app_id # https://app.spatialreal.ai/apps
    SPATIALREAL_AVATAR_ID=6aed28f9-674c-4ffb-89ee-b447b28aa3ed # https://app.spatialreal.ai/avatars/library
    ```

    Create `backend/pyproject.toml` to install the Python dependencies used in this guide.
    Use these versions to match the quickstart runtime behavior.

    ```toml title="backend/pyproject.toml" theme={null}
    [project]
    name = "spatialreal-agent-quickstart-backend"
    version = "0.1.0"
    requires-python = ">=3.10,<3.15"
    dependencies = [
      "flask>=3.0.0",
      "flask-cors>=4.0.0",
      "python-dotenv>=1.0.0",
      "livekit-api>=1.1.0",
      "livekit-agents==1.4.4",
      "livekit-plugins-google==1.4.4",
      "livekit-plugins-spatialreal==1.4.4",
    ]
    ```

    Create `backend/token_server.py` to generate LiveKit JWTs and dispatch the agent to the room.

    ```python title="backend/token_server.py" theme={null}
    import asyncio
    import os
    from datetime import timedelta
    from uuid import uuid4

    from dotenv import load_dotenv
    from flask import Flask, jsonify, request
    from flask_cors import CORS
    from livekit import api

    load_dotenv()

    app = Flask(__name__)
    CORS(app)

    LIVEKIT_URL = os.getenv("LIVEKIT_URL")
    LIVEKIT_API_KEY = os.getenv("LIVEKIT_API_KEY")
    LIVEKIT_API_SECRET = os.getenv("LIVEKIT_API_SECRET")


    async def create_room_and_dispatch(room_name: str) -> None:
        lkapi = api.LiveKitAPI(LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET)
        try:
            try:
                await lkapi.room.create_room(api.CreateRoomRequest(name=room_name))
            except Exception:
                # Room may already exist.
                pass

            await lkapi.agent_dispatch.create_dispatch(
                api.CreateAgentDispatchRequest(room=room_name, agent_name="voice-assistant")
            )
        finally:
            await lkapi.aclose()


    @app.route("/token", methods=["POST"])
    def token():
        if not LIVEKIT_API_KEY or not LIVEKIT_API_SECRET:
            return jsonify({"error": "LiveKit credentials not configured"}), 500

        body = request.get_json() or {}
        room_name = body.get("room", "voice-agent-room")
        requested_identity = body.get("identity")
        identity = (
            requested_identity.strip()
            if isinstance(requested_identity, str) and requested_identity.strip()
            else f"browser-{uuid4().hex[:8]}"
        )

        jwt = (
            api.AccessToken(LIVEKIT_API_KEY, LIVEKIT_API_SECRET)
            .with_identity(identity)
            .with_name(identity)
            .with_ttl(timedelta(hours=1))
            .with_grants(
                api.VideoGrants(
                    room_join=True,
                    room=room_name,
                    can_publish=True,
                    can_subscribe=True,
                    can_publish_data=True,
                )
            )
            .to_jwt()
        )

        try:
            asyncio.run(create_room_and_dispatch(room_name))
        except Exception as exc:
            # Keep returning token so frontend can still connect for debugging.
            print(f"Warning: Failed to dispatch agent: {exc}")

        return jsonify(
            {
                "token": jwt,
                "url": LIVEKIT_URL,
                "room": room_name,
                "identity": identity,
            }
        )


    if __name__ == "__main__":
        app.run(host="0.0.0.0", port=8080, debug=True)
    ```

    Create `backend/agent.py` to run the Gemini Live voice agent and start the SpatialReal avatar session.
    After this worker joins the room, user microphone audio is processed in real time.

    ```python title="backend/agent.py" theme={null}
    import os

    from dotenv import load_dotenv
    from livekit.agents import Agent, AgentSession, AutoSubscribe, JobContext, WorkerOptions, cli
    from livekit.plugins import google
    from livekit.plugins.spatialreal import AvatarSession

    load_dotenv()


    class VoiceAssistant(Agent):
        def __init__(self) -> None:
            super().__init__(
                instructions="You are a helpful voice assistant. Keep replies short and natural."
            )


    async def entrypoint(ctx: JobContext) -> None:
        await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)

        session = AgentSession(
            llm=google.realtime.RealtimeModel(
                model=os.getenv("E2E_GOOGLE_MODEL", "gemini-2.5-flash"),
                voice=os.getenv("E2E_GOOGLE_VOICE", "Puck"),
                api_key=os.getenv("GOOGLE_API_KEY"),
            )
        )

        avatar = AvatarSession()
        await avatar.start(session, room=ctx.room)

        await session.start(agent=VoiceAssistant(), room=ctx.room)


    if __name__ == "__main__":
        cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint, agent_name="voice-assistant"))
    ```
  </Step>

  <Step title="Configure frontend">
    Install packages, then add these frontend files exactly as shown.

    Run this install command in `frontend` to add AvatarKit RTC and a compatible LiveKit client version.
    This prepares the frontend runtime before you add config and UI code.

    ```bash theme={null}
    cd frontend
    pnpm install
    pnpm add @spatialwalk/avatarkit @spatialwalk/avatarkit-rtc livekit-client@2.16.1
    ```

    Create `frontend/.env` with your public avatar IDs and token endpoint.
    For production, set `VITE_TOKEN_ENDPOINT` to your deployed backend URL.

    ```bash title="frontend/.env" theme={null}
    VITE_SPATIALREAL_APP_ID=your_app_id # https://app.spatialreal.ai/apps
    VITE_SPATIALREAL_AVATAR_ID=6aed28f9-674c-4ffb-89ee-b447b28aa3ed # https://app.spatialreal.ai/avatars/library
    VITE_TOKEN_ENDPOINT=http://localhost:8080/token # Your backend token server URL
    ```

    Update `frontend/vite.config.ts` to enable AvatarKit build support.
    The `/token` proxy routes local frontend requests to your backend token server.

    ```ts title="frontend/vite.config.ts" theme={null}
    import { defineConfig } from 'vite'
    import vue from '@vitejs/plugin-vue'
    import { avatarkitVitePlugin } from '@spatialwalk/avatarkit/vite'

    export default defineConfig({
      plugins: [vue(), avatarkitVitePlugin()],
      server: {
        port: 3000,
        proxy: {
          '/token': {
            target: 'http://localhost:8080',
            changeOrigin: true,
          },
        },
      },
    })
    ```

    Create `frontend/src/App.vue` with a minimal video-call layout and two controls.
    This component mounts the avatar canvas, connects to LiveKit, and toggles microphone publishing.

    ```vue title="frontend/src/App.vue" theme={null}
    <script setup lang="ts">
    import { nextTick, onBeforeUnmount, ref } from 'vue'
    import { Room } from 'livekit-client'
    import {
      AvatarManager,
      AvatarSDK,
      AvatarView,
      DrivingServiceMode,
      Environment,
    } from '@spatialwalk/avatarkit'
    import { AvatarPlayer, LiveKitProvider } from '@spatialwalk/avatarkit-rtc'

    const container = ref<HTMLDivElement | null>(null)
    const status = ref('Click Connect to start')
    const connecting = ref(false)
    const connected = ref(false)
    const micOn = ref(false)

    let avatarView: AvatarView | null = null
    let player: AvatarPlayer | null = null
    let roomClient: Room | null = null

    async function connect(): Promise<void> {
      if (connecting.value || connected.value) return
      connecting.value = true
      status.value = 'Requesting token...'

      try {
        const response = await fetch(import.meta.env.VITE_TOKEN_ENDPOINT || '/token', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify({ room: 'voice-agent-room' }),
        })
        if (!response.ok) throw new Error('Failed to fetch token')

        const { token, url, room } = await response.json()

        if (!AvatarSDK.isInitialized) {
          await AvatarSDK.initialize(import.meta.env.VITE_SPATIALREAL_APP_ID, {
            environment: Environment.intl,
            drivingServiceMode: DrivingServiceMode.host,
          })
        }

        await nextTick()
        const mountEl = container.value
        if (!mountEl) throw new Error('Avatar container is not ready')

        await player?.disconnect().catch(() => undefined)
        avatarView?.dispose()

        const avatar = await AvatarManager.shared.load(import.meta.env.VITE_SPATIALREAL_AVATAR_ID)
        avatarView = new AvatarView(avatar, mountEl)
        player = new AvatarPlayer(new LiveKitProvider(), avatarView)

        status.value = 'Connecting to LiveKit...'
        await player.connect({ url, token, roomName: room })
        roomClient = player.getNativeClient() as Room
        if (!roomClient) throw new Error('LiveKit room client is unavailable')

        await roomClient.startAudio()

        connected.value = true
        status.value = 'Connected. Click Start Mic to talk.'
      } catch (error) {
        status.value = error instanceof Error ? error.message : 'Connection failed'
      } finally {
        connecting.value = false
      }
    }

    async function startMic(): Promise<void> {
      if (!player || micOn.value) return

      status.value = 'Starting microphone...'
      try {
        await player.startPublishing()
        micOn.value = true
        status.value = 'Mic is on. Start speaking.'
      } catch (error) {
        status.value = error instanceof Error ? `Failed to start microphone: ${error.message}` : 'Failed to start microphone.'
      }
    }

    async function stopMic(): Promise<void> {
      if (!micOn.value) return
      await player?.stopPublishing().catch(() => undefined)
      micOn.value = false
      status.value = 'Mic is off.'
    }

    async function disconnect(): Promise<void> {
      await stopMic()
      await player?.disconnect().catch(() => undefined)
      avatarView?.dispose()
      player = null
      avatarView = null
      roomClient = null
      connected.value = false
      status.value = 'Disconnected'
    }

    async function toggleConnection(): Promise<void> {
      if (connecting.value) return
      if (connected.value) {
        await disconnect()
        return
      }
      await connect()
    }

    async function toggleMic(): Promise<void> {
      if (!connected.value || connecting.value) return
      if (micOn.value) {
        await stopMic()
        return
      }
      await startMic()
    }

    onBeforeUnmount(async () => {
      await disconnect()
    })
    </script>

    <template>
      <div style="min-height:100vh; display:flex; align-items:center; justify-content:center; padding:16px;">
        <div style="width:min(720px, 100%); display:flex; flex-direction:column; gap:10px;">
          <div ref="container" style="width:100%; aspect-ratio:16/10; min-height:320px; border-radius:12px; overflow:hidden; border:1px solid;" />

          <div style="display:flex; gap:8px; flex-wrap:wrap;">
            <button :disabled="connecting" @click="toggleConnection">
              {{ connecting ? 'Connecting...' : connected ? 'Disconnect' : 'Connect' }}
            </button>
            <button :disabled="!connected || connecting" @click="toggleMic">
              {{ micOn ? 'Stop Mic' : 'Start Mic' }}
            </button>
          </div>

          <div style="font-size:14px;">{{ status }}</div>
        </div>
      </div>
    </template>
    ```
  </Step>

  <Step title="Verify project structure">
    Before running, confirm your files match this tree.
    This quick check helps catch missing or misplaced files before launch.

    ```text title="SpatialReal Avatar quickstart" theme={null}
    spatialreal-agent-quickstart/
    ├── backend/
    │   ├── .env
    │   ├── pyproject.toml
    │   ├── token_server.py
    │   └── agent.py
    └── frontend/
        ├── .env
        ├── vite.config.ts
        └── src/
            └── App.vue
    ```
  </Step>

  <Step title="Install and run">
    Run these commands in three terminals, then open `http://localhost:3000`, click **Connect**, and then click **Start Mic**.

    Start the token server first so frontend can fetch JWTs.

    ```bash theme={null}
    # Terminal 1
    cd spatialreal-agent-quickstart/backend
    uv sync
    uv run token_server.py
    ```

    Start the agent worker second so it can join the room when dispatched.

    ```bash theme={null}
    # Terminal 2
    cd spatialreal-agent-quickstart/backend
    uv run agent.py dev
    ```

    Start the frontend last, then open the local URL and begin speaking.

    ```bash theme={null}
    # Terminal 3
    cd spatialreal-agent-quickstart/frontend
    pnpm dev
    ```
  </Step>
</Steps>

## What Happens

* Frontend requests `/token`
* Backend returns LiveKit JWT and dispatches `voice-assistant`
* Agent joins room with end-to-end realtime model
* `AvatarSession` publishes avatar output
* `AvatarPlayer` renders avatar and streams your microphone audio

## More Implementations

<Card title="AvatarKit Voice Agent Demo" icon="github" href="https://github.com/spatialwalk/avatarkit-voice-agent-demo">
  A full reference repository with more implementation details, including different frontend UI options and multiple backend agent patterns.
</Card>


# Changelog
Source: https://docs.spatialreal.ai/overview/changelog



<Update label="1.0.0-beta.80" description="2025-01">
  **Next.js Plugin** - Added `withAvatarkit` wrapper for Next.js projects to automate WASM file configuration with webpack

  * Patches Emscripten `scriptDirectory` for correct asset resolution
  * Copies WASM files to `static/chunks/` automatically
  * Adds `application/wasm` Content-Type headers
  * Preserves existing webpack and headers configurations
</Update>

<Update label="1.0.0-beta.75" description="2025-01">
  **Local Log Upload** - Automatic local log upload for debugging and troubleshooting
</Update>

<Update label="1.0.0-beta.71" description="2024-12">
  **API Refinement** - Improved API structure by marking internal classes with `@internal` for better code organization

  **Type Safety** - Enhanced type definitions by hiding internal implementation details from public API
</Update>

<Update label="1.0.0-beta.64" description="2024-12">
  **Vite Plugin** - Added `avatarkitVitePlugin` to automate WASM file configuration

  * Automatically handles WASM MIME types in dev server
  * Copies WASM files during build
  * Generates `_headers` file for Cloudflare Pages
</Update>

<Update label="1.0.0-beta.63" description="2024-12">
  **Audio Context API** - Added `initializeAudioContext()` method that must be called in user gesture context

  * Ensures AudioContext is created properly to satisfy browser security policies
  * All audio operations now require prior initialization
</Update>

<Update label="1.0.0-beta.55" description="2024-11">
  **Eye Tracking Control** - Added `eyeTrackingEnabled` parameter to `PostProcessingConfig` for real-time control

  **Point Count API** - Added `getPointCount()` method to `AvatarController` to get avatar point cloud count
</Update>

<Update label="1.0.0-beta.51" description="2024-11">
  **Post-Processing Enhancement** - Added `translation` and `eyePose` fields to `PostProcessingConfig`

  **Bugfix** - Fixed avatar settings update issue when using cached avatars
</Update>

<Update label="1.0.0-beta.45" description="2024-10">
  **Immediate Rendering** - Post-processing and camera updates apply immediately when playback is paused

  **API Change** - Replaced `ready` Promise with `onFirstRendering` callback for better async handling
</Update>

<Update label="1.0.0-beta.36" description="2024-10">
  **Audio Configuration** - Added configurable audio sample rate support (8000-48000 Hz, default: 16000)

  * Audio recording and playback automatically use configured sample rate
</Update>

<Update label="1.0.0-beta.35" description="2024-10">
  **API Change** - Changed `setTransform()` method to `transform` getter/setter property
</Update>

<Update label="1.0.0-beta.29" description="2024-09">
  **Breaking Changes** - Removed `Environment.test`, changed default log level to `off`

  **New Features** - Added avatar caching methods (`retrieve`, `clear`, `clearAll`)

  **Background Image** - Added background image control in `AvatarView`
</Update>

<Update label="1.0.0-beta.28" description="2024-09">
  **Breaking Change** - Renamed `AvatarKit` to `AvatarSDK` for better consistency

  * Update imports: `AvatarKit` → `AvatarSDK`
</Update>

<Update label="1.0.0-beta.22" description="2024-08">
  **API Changes** - Renamed `onAvatarState` to `onConversationState`, `Environment.us` to `Environment.intl`

  **Volume Control** - Added `setVolume()` and `getVolume()` methods for audio playback control
</Update>

<Update label="1.0.0-beta.17" description="2024-08">
  **Fallback Mechanism** - Added automatic audio-only fallback when animation data is unavailable

  **API Change** - Moved playback mode configuration to `AvatarSDK.initialize()`
</Update>

<Update label="1.0.0-beta.3" description="2024-07">
  **Architecture** - Refactored to three-layer architecture (Rendering, Playback, Network)

  **Dual Mode** - Added SDK mode and Host mode support
</Update>

<Update label="temp" description="temp" />

<Update label="temp" description="temp" />

<Update label="temp" description="temp" />

<Update label="temp" description="temp" />

<Update label="temp" description="temp" />

<Update label="temp" description="temp" />

<Update label="temp" description="temp" />


# Demo Projects
Source: https://docs.spatialreal.ai/overview/demo-projects

Complete working examples for every platform

Get started quickly with our demo projects. Each repository contains a fully functional application you can run immediately.

<CardGroup>
  <Card title="Voice Agent + Avatar Demo" icon="microphone" href="https://github.com/spatialwalk/avatarkit-voice-agent-demo">
    Fully functioning voice agent + avatar example projects that run after filling API keys in `.env`.
  </Card>

  <Card title="Web Demo" icon="globe" href="https://github.com/spatialwalk/avatar-kit-web-demo">
    Vanilla JS, Vue, and React examples with Vite and Next.js configurations.
  </Card>

  <Card title="iOS Demo" icon="apple" href="https://github.com/spatialwalk/avatar-kit-ios-demo">
    Swift / SwiftUI example with SDK Mode integration.
  </Card>

  <Card title="Android Demo" icon="android" href="https://github.com/spatialwalk/avatar-kit-android-demo">
    Kotlin / Compose example with SDK Mode integration.
  </Card>
</CardGroup>

## Voice Agent + Avatar

**Repository:** [github.com/spatialwalk/avatarkit-voice-agent-demo](https://github.com/spatialwalk/avatarkit-voice-agent-demo)

A fully functioning voice agent + avatar demo collection for real-time interaction.

* End-to-end voice agent + avatar pipeline
* Multiple runnable example projects
* Works out of the box after filling required API keys in `.env`

```bash theme={null}
git clone https://github.com/spatialwalk/avatarkit-voice-agent-demo.git
cd avatarkit-voice-agent-demo
```

## Web

**Repository:** [github.com/spatialwalk/avatar-kit-web-demo](https://github.com/spatialwalk/avatar-kit-web-demo)

Includes multiple framework examples:

| Example | Framework               | Build Tool     |
| ------- | ----------------------- | -------------- |
| Vanilla | Plain TypeScript        | Vite           |
| Vue     | Vue 3 + Composition API | Vite           |
| React   | React 18+               | Vite / Next.js |

```bash theme={null}
git clone https://github.com/spatialwalk/avatar-kit-web-demo.git
cd avatar-kit-web-demo
pnpm install
pnpm dev
```

## iOS

**Repository:** [github.com/spatialwalk/avatar-kit-ios-demo](https://github.com/spatialwalk/avatar-kit-ios-demo)

* Swift 6.2 / SwiftUI
* iOS 16+
* Metal rendering

```bash theme={null}
git clone https://github.com/spatialwalk/avatar-kit-ios-demo.git
```

Open the `.xcodeproj` in Xcode, set your App ID and Session Token, then run on a physical device.

<Warning>
  Metal rendering is not supported in the iOS Simulator. Use a physical device for testing.
</Warning>

## Android

**Repository:** [github.com/spatialwalk/avatar-kit-android-demo](https://github.com/spatialwalk/avatar-kit-android-demo)

* Kotlin / Jetpack Compose
* API 24+
* Vulkan rendering

```bash theme={null}
git clone https://github.com/spatialwalk/avatar-kit-android-demo.git
```

Open in Android Studio, set your App ID and Session Token, then run on a physical device or emulator with Vulkan support.


# FAQ
Source: https://docs.spatialreal.ai/overview/faq

Frequently asked questions

### Common Issues

<AccordionGroup>
  <Accordion title="Supported Audio Format">
    Currently only support PCM 16-bit mono audio at multiple sample rates:
    8000, 16000, 22050, 24000, 32000, 44100, 48000.
  </Accordion>

  <Accordion title="Avatar Playback Fallback Strategy">
    If the avatar animation drive data is abnormal after passing in audio, the fallback playback strategy will be triggered:

    * Audio plays normally
    * Animation plays idle breathing state / Animation stops
  </Accordion>
</AccordionGroup>

## Performance Issues

<AccordionGroup>
  <Accordion title="Avatar Rendering Resolution Support">
    Our avatar models don't use the concept of resolution.

    We can reconstruct 4K resolution 3D avatar models, but this depends on the model's point count.

    Fewer points result in less detail restoration, but all models can be rendered to the corresponding resolution on the client side.

    Metrics such as PSNR and LPIPS are typically used to measure the quality of detail restoration.
  </Accordion>

  <Accordion title="Can the SDK dynamically switch between different models based on device performance?">
    Not currently supported. Developers need to implement their own strategy for switching between different models based on device performance.
  </Accordion>
</AccordionGroup>


# Get Your API Keys
Source: https://docs.spatialreal.ai/overview/get-apikeys

To integrate with SpatialReal, first create your API Key.

Follow these steps to get your `App ID` and `API Key`.

<Steps>
  <Step title="Open API Keys page">
    Go to [https://app.spatialreal.ai/apps](https://app.spatialreal.ai/apps).
  </Step>

  <Step title="Click Create API Key">
    On the API key page, click **Create API Key**.

    <Frame>
      <img alt="API Keys page" />
    </Frame>
  </Step>

  <Step title="Enter application name">
    Type your **Application Name** and click **Create**.

    <Frame>
      <img alt="Click Create API Key" />
    </Frame>
  </Step>

  <Step title="Copy App ID and API Key">
    In the popup, copy both **App ID** and **API Key**.

    <Frame>
      <img alt="Create app and view popup" />
    </Frame>
  </Step>
</Steps>


# How to integrate
Source: https://docs.spatialreal.ai/overview/integration_overview

Choose an integration mode and start from runnable example projects

<Info>
  SpatialReal currently provides **avatar-only** services, focusing on generating and rendering real-time avatar animations based on audio input. Voice conversation logic, speech synthesis, and other agent functionalities are managed by your application or third-party services.
</Info>

## Compare Integration Modes

| Mode                                                            | Characteristic               | Latency     | Dev Effort | Ideal Scenario                                                                                |
| --------------------------------------------------------------- | ---------------------------- | ----------- | ---------- | --------------------------------------------------------------------------------------------- |
| **[SDK Mode](/concepts/integrations#sdk-mode)**                 | Client-centric               | Moderate ⏱️ | Low 🟢     | Easy client-centric integration.                                                              |
| **[Framework Plugin](/concepts/integrations#framework-plugin)** | Voice agent framework plugin | Ultra-Low ⚡ | Low 🟢     | Use [LiveKit Agents](https://docs.livekit.io/agents/) or [TEN Framework](https://theten.ai/). |
| **[RTC Mode](/concepts/integrations#rtc-mode)**                 | Transport via Agora/LiveKit  | Ultra-Low ⚡ | Medium 🟡  | You want to optimize latency, and use LiveKit or Agora as transport layer                     |
| **[Host Mode](/concepts/integrations#host-mode)**               | Use your own transport layer | Low 💨      | High 🔴    | Full control over data transport.                                                             |

## Start from Runnable Examples

<Card title="AvatarKit Voice Agent Demo" icon="code" href="https://github.com/spatialwalk/avatarkit-voice-agent-demo/tree/main/livekit-agents">
  A collection of example projects that can work out of the box after API keys are filled in `.env`.
</Card>

```bash theme={null}
git clone https://github.com/spatialwalk/avatarkit-voice-agent-demo.git
cd avatarkit-voice-agent-demo
```

## Guides

<CardGroup>
  <Card title="SDK Mode Guide" icon="book-open" href="/guide/sdk-mode">
    Start with client-centric integration.
  </Card>

  <Card title="Host Mode Guide" icon="book-open" href="/guide/host-mode">
    Build custom transport and server orchestration.
  </Card>
</CardGroup>

<Card title="RTC Mode & Framework Plugin Guide" icon="book-open" href="/guide/rtc-mode">
  Use LiveKit or Agora for real-time delivery, including LiveKit Agents and TEN Framework integration.
</Card>

## Want to Learn More?

Read the full [Integration Modes](/concepts/integrations) doc to understand the technical details, service architecture, and best practices for each integration mode.


# Welcome to SpatialReal Documentation
Source: https://docs.spatialreal.ai/overview/introduction



SpatialReal is the next-generation AI infrastructure for empathetic human-AI interactions. We build **real-time**, **high-fidelity**, and **cost-effective** digital avatar interactions through on-device rendering technology.

## Why SpatialReal?

Traditional digital avatar solutions force painful trade-offs between cost, visual quality, and real-time performance. SpatialReal solves this trilemma:

| Metric            | Traditional Cloud Rendering  | SpatialReal On-Device          |
| ----------------- | ---------------------------- | ------------------------------ |
| **Bandwidth**     | 1-2 MB/s (Video Stream)      | 10-20 KB/s (Driver Data)       |
| **Cost**          | High GPU rental costs        | Ultra-low edge compute         |
| **Latency**       | > 3s                         | \< 1.5s                        |
| **Compatibility** | High-speed network dependent | 99% of Android/iOS/Web devices |

## Use Cases

* **Online Education** - Immersive 1-on-1 tutoring with precise lip-sync for language learning and K12
* **Recruitment & Training** - 7x24h AI interviewers and roleplay training simulations
* **Customer Service** - Brand ambassadors with visual eye contact that builds emotional connection

## Creating an account

### Sign Up

Visit [SpatialReal Studio](https://app.spatialreal.ai/) to create your free account.

### SpatialReal Studio

[SpatialReal Studio](https://app.spatialreal.ai/) is your control center where you can:

* Free access to a large avatar library
* Manage your API keys
* Monitor your usage
* Control your subscription

## Quick Start

<CardGroup>
  <Card title="Use LiveKit(recommended)" icon="play" href="/overview/agent">
    LiveKit is a realtime communication platform that allows you to create voice and video chat applications. Deploying your own LiveKit server is quick and easy.
  </Card>

  <Card title="Use Avatarkit SDKs" icon="key" href="/overview/speech-to-avatar">
    For when you need full control of your tech stack (e.g., custom voice bots).
  </Card>
</CardGroup>

<Card title="Start building in 5 minutes" icon="rocket" href="/overview/integration_overview">
  Compare integration modes and start from runnable example projects.
</Card>

## About Us

Learn more about SpatialReal and our mission at [spatialreal.ai](https://www.spatialreal.ai/).


# Speech-to-Avatar
Source: https://docs.spatialreal.ai/overview/speech-to-avatar

Rapidly experience SpatialReal Avatar with Avatarkit Web SDK

Use this quickstart to create a minimal **Speech-to-Avatar** web demo with SpatialReal SDK Mode.
In a few steps, you will load your avatar, send speech from a downloadable audio link, and render a realistic talking-avatar experience directly in the browser.

<Note>
  You can just copy each code block as-is and replace values in `.env`.
  Or just dump this page to any LLM you like and ask it to implement this demo for you.
</Note>

<Card title="Github Repository of this quickstart" icon="github" href="https://github.com/spatialwalk/avatarkit-voice-agent-demo/tree/main/spatialreal-speech-to-avatar-quickstart" />

## Prerequisites

* `SPATIALREAL_APP_ID`
* `SPATIALREAL_AVATAR_ID`
* `SESSION_TOKEN`
* Node.js 18+, `pnpm`

<Steps>
  <Step title="Create workspace">
    ```bash theme={null}
    mkdir spatialreal-speech-to-avatar-quickstart
    cd spatialreal-speech-to-avatar-quickstart
    pnpm dlx create-vite@latest . --template vue-ts
    ```
  </Step>

  <Step title="Install dependencies">
    ```bash theme={null}
    pnpm install
    pnpm add @spatialwalk/avatarkit
    ```
  </Step>

  <Step title="Configure environment">
    In [SpatialReal Studio](https://app.spatialreal.ai/app), generate a [temporary session token](/studio/aki-key#temporary-session-token) and get your app ID, then fill in the values in `.env`.

    Create `.env`:

    ```bash title=".env" theme={null}
    VITE_SPATIALREAL_AVATAR_ID=6aed28f9-674c-4ffb-89ee-b447b28aa3ed # For more avatars, visit https://app.spatialreal.ai/avatars/library
    VITE_SPATIALREAL_APP_ID=your_app_id # https://app.spatialreal.ai/apps
    VITE_SPATIALREAL_SESSION_TOKEN=your_temporary_session_token # https://app.spatialreal.ai/apps -> details -> Generate Temporary Token
    ```
  </Step>

  <Step title="Update Vite config">
    ```ts title="vite.config.ts" theme={null}
    import { defineConfig } from 'vite'
    import vue from '@vitejs/plugin-vue'
    import { avatarkitVitePlugin } from '@spatialwalk/avatarkit/vite'

    export default defineConfig({
      plugins: [vue(), avatarkitVitePlugin()],
      server: { port: 3000 },
    })
    ```
  </Step>

  <Step title="Create app page">
    Create `src/App.vue` and paste the following snippets in order.

    Start with the base `<script setup>` structure. This part imports Vue and SpatialReal SDK APIs, reads values from `.env`, and defines core UI state.

    ```vue title="src/App.vue" theme={null}
    <script setup lang="ts">
    import { nextTick, onBeforeUnmount, ref } from 'vue'
    import {
      AvatarManager,
      AvatarSDK,
      AvatarView,
      DrivingServiceMode,
      Environment,
    } from '@spatialwalk/avatarkit'

    // UI state and avatar mount point.
    const container = ref<HTMLDivElement | null>(null)
    const status = ref('Click Connect Avatar to start')
    const connecting = ref(false)
    const sending = ref(false)

    // Active avatar view instance (created after avatar is loaded).
    let avatarView: AvatarView | null = null
    const connected = ref(false)

    // Runtime configuration from .env.
    const appId = import.meta.env.VITE_SPATIALREAL_APP_ID
    const avatarId = import.meta.env.VITE_SPATIALREAL_AVATAR_ID
    const sessionToken = import.meta.env.VITE_SPATIALREAL_SESSION_TOKEN

    if (!sessionToken) throw new Error('Missing VITE_SPATIALREAL_SESSION_TOKEN')
    ```

    Then add the audio loading function. This part fetches demo PCM audio from a fixed public URL and returns raw bytes for sending.

    ```ts title="src/App.vue" theme={null}
    // Download remote PCM audio as raw bytes.
    async function downloadPcmAudio(url: string): Promise<ArrayBuffer> {
      const response = await fetch(url)
      if (!response.ok) throw new Error('Failed to load demo audio URL')
      return response.arrayBuffer()
    }

    ```

    Next add avatar lifecycle and action logic. Use two explicit actions: one button to connect avatar, one button to send audio.

    ```ts title="src/App.vue" theme={null}
    // Release current connection and rendering resources before re-run/unmount.
    async function disposeAvatar(): Promise<void> {
      connected.value = false
      avatarView?.controller.close()
      avatarView?.dispose()
      avatarView = null
    }

    async function connectAvatar(): Promise<void> {
      if (connecting.value || connected.value) return
      connecting.value = true

      try {
        status.value = 'Using session token from .env...'

        if (!AvatarSDK.isInitialized) {
          await AvatarSDK.initialize(appId, {
            environment: Environment.intl,
            drivingServiceMode: DrivingServiceMode.sdk,
          })
        }
        AvatarSDK.setSessionToken(sessionToken)

        await nextTick()
        const mountEl = container.value
        if (!mountEl) throw new Error('Avatar container is not ready')

        if (!avatarView) {
          status.value = 'Loading avatar...'
          const avatar = await AvatarManager.shared.load(avatarId)
          avatarView = new AvatarView(avatar, mountEl)
          avatarView.controller.onConnectionState = (state) => {
            connected.value = state === 'connected'
          }
        }

        const controller = avatarView?.controller
        if (!controller) throw new Error('Avatar controller is not ready')

        status.value = 'Connecting to SpatialReal...'
        await controller.initializeAudioContext()
        await controller.start()
        await new Promise((resolve) => setTimeout(resolve, 300))
        if (!connected.value) throw new Error('Failed to connect to animation channel')

        status.value = 'Avatar connected. Click Send Audio.'
      } catch (error) {
        status.value = error instanceof Error ? error.message : 'Failed to run demo'
      } finally {
        connecting.value = false
      }
    }

    async function sendAudio(): Promise<void> {
      if (sending.value) return
      if (!connected.value || !avatarView) {
        status.value = 'Please click Connect Avatar first.'
        return
      }

      sending.value = true

      try {
        status.value = 'Downloading demo PCM audio...'
        const audioData = await downloadPcmAudio('https://cdn.spatialwalk.cloud/public/website/quickstart_voice.pcm')

        status.value = 'Sending audio...'
        avatarView.controller.send(audioData, true)
        status.value = `Audio sent (${audioData.byteLength} bytes)`
      } catch (error) {
        status.value = error instanceof Error ? error.message : 'Failed to send audio'
      } finally {
        sending.value = false
      }
    }

    onBeforeUnmount(async () => {
      await disposeAvatar()
    })
    </script>
    ```

    Finally add the template section. It renders the avatar container, separate connect/send buttons, and a status line.

    ```vue title="src/App.vue" theme={null}
    <template>
      <div style="min-height:100vh; display:flex; align-items:center; justify-content:center; padding:16px;">
        <div style="width:min(720px, 100%); display:flex; flex-direction:column; gap:10px;">
          <div ref="container" style="width:100%; aspect-ratio:16/10; min-height:320px; border-radius:12px; overflow:hidden; border:1px solid;" />

          <div style="display:flex; gap:8px; flex-wrap:wrap;">
            <button :disabled="connecting || connected" @click="connectAvatar">
              {{ connecting ? 'Connecting...' : connected ? 'Avatar Connected' : 'Connect Avatar' }}
            </button>
            <button :disabled="sending || !connected" @click="sendAudio">
              {{ sending ? 'Sending...' : 'Send Audio' }}
            </button>
          </div>

          <div style="font-size:14px;">{{ status }}</div>
        </div>
      </div>
    </template>
    ```
  </Step>

  <Step title="Verify project structure">
    ```text title="Speech-to-Avatar quickstart" theme={null}
    spatialreal-quickstart/
    ├── .env
    ├── vite.config.ts
    └── src/
        └── App.vue
    ```
  </Step>

  <Step title="Install and run">
    Run the app:

    ```bash theme={null}
    cd spatialreal-quickstart
    pnpm dev
    ```

    Open `http://localhost:3000`, click **Connect Avatar**, then click **Send Audio**.
  </Step>
</Steps>

## What Happens

* Frontend initializes AvatarKit in SDK Mode and connects avatar first
* Frontend reuses the same connection for repeated audio sends
* Frontend sets the session token directly from `.env`
* Frontend downloads demo PCM audio
* Downloaded PCM data is sent in one call


# Error Codes
Source: https://docs.spatialreal.ai/resources/error-codes

Reference links and troubleshooting guidance for SDK error handling

This page collects the main error-handling references for AvatarKit server and client SDKs.

## Server SDK

For backend integrations, use your server SDK's structured error information as the main reference.

* [Server SDK reference](/sdk-reference/python-sdk/python-sdk)

Server-side integrations should account for session token failures, WebSocket handshake failures, runtime server errors, and unexpected connection closures.

### Common server SDK error codes

These stable error codes are useful when building retry, logging, and alerting flows on your backend.

| Error code                | Meaning                                                 |
| ------------------------- | ------------------------------------------------------- |
| `sessionTokenExpired`     | Session token expired or is no longer authorized        |
| `sessionTokenInvalid`     | Session token is invalid or empty                       |
| `appIDUnrecognized`       | App ID is not recognized by the server                  |
| `appIDMismatch`           | Session token belongs to a different app                |
| `avatarNotFound`          | Requested avatar does not exist                         |
| `billingRequired`         | Session is blocked by billing requirements              |
| `creditsExhausted`        | Runtime or connect-time credits are exhausted           |
| `sessionDurationExceeded` | Session hit a billing-enforced duration limit           |
| `unsupportedSampleRate`   | Audio sample rate is not accepted by the handshake      |
| `invalidEgressConfig`     | LiveKit or Agora egress configuration is invalid        |
| `egressUnavailable`       | Egress service is unavailable or not configured         |
| `idleTimeout`             | Session was closed because no input was received        |
| `upstreamError`           | Upstream internal service failed                        |
| `protocolError`           | Invalid protobuf payload or unexpected message sequence |
| `connectionFailed`        | Transport-level connection setup failed                 |
| `connectionClosed`        | WebSocket closed unexpectedly                           |
| `serverError`             | Server returned an unclassified error                   |
| `invalidRequest`          | Client request payload or parameters are invalid        |
| `unknown`                 | Fallback code when the SDK cannot classify the failure  |

### Server SDK error fields

When handling server SDK errors, check these fields to determine the correct recovery path:

* `code`: stable SDK error code for programmatic handling
* `phase`: where the failure happened, such as `session_token`, `websocket_connect`, or `websocket_runtime`
* `http_status`: HTTP status for token or upgrade failures
* `server_code`, `server_title`, `server_detail`: normalized details returned by the server
* `connection_id`, `req_id`: correlation IDs for tracing requests in logs
* `close_code`, `close_reason`: WebSocket close details for unexpected disconnects

## Client SDK

For client-side troubleshooting, use the SDK-specific API and FAQ pages below.

### Web SDK

* [Web SDK API reference](/sdk-reference/web-sdk/api-reference)
* [Web SDK FAQ](/sdk-reference/web-sdk/faq)

The Web SDK API reference includes a dedicated [Error Codes section](/sdk-reference/web-sdk/api-reference#error-codes) for controller and runtime failures.

### iOS SDK

* [iOS SDK API reference](/sdk-reference/ios-sdk/api-reference)
* [iOS SDK FAQ](/sdk-reference/ios-sdk/faq)

Use the `AvatarController.onError` callback documented in the API reference to capture runtime failures and surface them in your app.

### Android SDK

* [Android SDK API reference](/sdk-reference/android-sdk/api-reference)
* [Android SDK FAQ](/sdk-reference/android-sdk/faq)

Use the `AvatarController.onError` callback documented in the API reference to handle rendering, networking, and playback failures on Android.

## Recommended handling flow

1. Log the SDK error code and phase.
2. Capture server correlation identifiers when available.
3. Retry only transient failures such as connection interruptions or timeouts.
4. Surface configuration and authentication errors to operators with the linked SDK references.


# API Reference
Source: https://docs.spatialreal.ai/sdk-reference/android-sdk/api-reference

Browse the complete API reference

### AvatarSDK

The core management class of the SDK, responsible for initialization and global configuration.

```kotlin theme={null}
object AvatarSDK
```

##### Properties

<Accordion title="sessionToken">
  The session token used to authenticate avatars with the AvatarKit Server.

  ```kotlin theme={null}
  var sessionToken: String
  ```
</Accordion>

<Accordion title="userId">
  The user identifier.

  ```kotlin theme={null}
  var userId: String
  ```
</Accordion>

<Accordion title="version">
  Returns the version of AvatarKit.

  ```kotlin theme={null}
  val version: String
  ```
</Accordion>

##### Methods

<Accordion title="initialize(context, appId, configuration)">
  Initialize AvatarKit.

  ```kotlin theme={null}
  fun initialize(
      context: Context,
      appId: String,
      configuration: Configuration
  )
  ```

  **Parameters:**

  * `context`: Application context
  * `appId`: Your application identifier
  * `configuration`: The configuration for AvatarKit
</Accordion>

<Accordion title="supportsCurrentDevice()">
  Returns a Boolean value that indicates whether AvatarKit supports the current device.

  ```kotlin theme={null}
  fun supportsCurrentDevice(): Boolean
  ```
</Accordion>

<Accordion title="benchmark()">
  Measures the device's computational performance for avatar rendering.

  ```kotlin theme={null}
  fun benchmark(): Int
  ```

  **Returns:** A performance score indicating the device's capability.
</Accordion>

### AvatarManager

Avatar resource manager, responsible for downloading, caching, and loading avatar data.

```kotlin theme={null}
object AvatarManager
```

##### Methods

<Accordion title="initialize(context)">
  Initialize AvatarManager. Must be called before use.

  ```kotlin theme={null}
  fun initialize(context: Context)
  ```

  **Parameters:**

  * `context`: Application context
</Accordion>

<Accordion title="load(id, onProgress)">
  Loads an avatar by ID.

  ```kotlin theme={null}
  suspend fun load(
      id: String,
      onProgress: ((LoadProgress) -> Unit)? = null
  ): Avatar
  ```

  **Parameters:**

  * `id`: The avatar identifier
  * `onProgress`: Optional progress callback

  **Returns:** The loaded `Avatar` instance.
</Accordion>

<Accordion title="clear(id)">
  Clears cached data for a specific avatar.

  ```kotlin theme={null}
  suspend fun clear(id: String)
  ```

  **Parameters:**

  * `id`: The avatar identifier to clear
</Accordion>

<Accordion title="clearAll()">
  Clears all cached avatar data.

  ```kotlin theme={null}
  suspend fun clearAll()
  ```
</Accordion>

<Accordion title="getCacheSize(id)">
  Gets the cache size for a specific avatar.

  ```kotlin theme={null}
  suspend fun getCacheSize(id: String): Long
  ```

  **Parameters:**

  * `id`: The avatar identifier

  **Returns:** The cache size in bytes.
</Accordion>

<Accordion title="getAllCacheSize()">
  Gets the total cache size for all avatars.

  ```kotlin theme={null}
  suspend fun getAllCacheSize(): Long
  ```

  **Returns:** The total cache size in bytes.
</Accordion>

<Accordion title="getCacheStats()">
  Get cache statistics.

  ```kotlin theme={null}
  suspend fun getCacheStats(): CacheStats
  ```

  **Returns:** A `CacheStats` object containing total entries, total size, and max size.
</Accordion>

### AvatarController

Real-time communication controller that handles WebSocket connections and audio/video data.

```kotlin theme={null}
class AvatarController
```

##### Properties

<Accordion title="onConnectionState">
  Callback for connection state changes.

  ```kotlin theme={null}
  var onConnectionState: ((ConnectionState) -> Unit)?
  ```
</Accordion>

<Accordion title="onConversationState">
  Callback for conversation state changes.

  ```kotlin theme={null}
  var onConversationState: ((ConversationState) -> Unit)?
  ```
</Accordion>

<Accordion title="onError">
  Callback for error events.

  ```kotlin theme={null}
  var onError: ((AvatarError) -> Unit)?
  ```
</Accordion>

<Accordion title="volume">
  The volume of playback (0.0 to 1.0).

  ```kotlin theme={null}
  var volume: Float
  ```
</Accordion>

##### Methods

<Accordion title="start()">
  Starts the avatar driving service connection.

  ```kotlin theme={null}
  fun start()
  ```
</Accordion>

<Accordion title="close()">
  Close connection.

  ```kotlin theme={null}
  fun close()
  ```
</Accordion>

<Accordion title="pauseRendering()">
  Pause the avatar view to pause rendering.

  ```kotlin theme={null}
  fun pauseRendering()
  ```
</Accordion>

<Accordion title="resumeRendering()">
  Resume the avatar view to resume rendering.

  ```kotlin theme={null}
  fun resumeRendering()
  ```
</Accordion>

<Accordion title="interrupt()">
  Stops playback and terminates the current conversation.

  ```kotlin theme={null}
  fun interrupt()
  ```
</Accordion>

<Accordion title="send(audioData, end)">
  Sends audio to the avatar driving service.

  ```kotlin theme={null}
  suspend fun send(audioData: ByteArray, end: Boolean = false): String
  ```

  **Parameters:**

  * `audioData`: The audio data to send
  * `end`: Whether this is the end of the audio stream

  **Returns:** A conversation ID string.
</Accordion>

<Accordion title="yield(audioData, end, audioFormat)">
  Yields audio from the host.

  ```kotlin theme={null}
  suspend fun yield(
      audioData: ByteArray,
      end: Boolean = false,
      audioFormat: AudioFormat? = null
  ): String
  ```

  **Parameters:**

  * `audioData`: The audio data
  * `end`: Whether this is the end of the audio stream
  * `audioFormat`: The audio format (optional)

  **Returns:** A conversation ID string.
</Accordion>

<Accordion title="yield(animations, reqId)">
  Yields animations from the host.

  ```kotlin theme={null}
  fun yield(animations: List<ByteArray>, reqId: String)
  ```

  **Parameters:**

  * `animations`: List of animation data
  * `reqId`: The request ID returned from yield audio
</Accordion>

### AvatarView

3D rendering view that automatically creates and manages AvatarController.

```kotlin theme={null}
class AvatarView : FrameLayout
```

##### Initializers

<Accordion title="constructor(context)">
  Creates a new avatar view.

  ```kotlin theme={null}
  constructor(context: Context)
  ```
</Accordion>

##### Properties

<Accordion title="avatarController">
  The controller for the avatar.

  ```kotlin theme={null}
  val avatarController: AvatarController?
  ```
</Accordion>

##### Methods

<Accordion title="init(avatar, scope)">
  Initialize view with avatar.

  ```kotlin theme={null}
  fun init(avatar: Avatar, scope: CoroutineScope)
  ```

  **Parameters:**

  * `avatar`: The avatar to display
  * `scope`: Coroutine scope, usually obtained via `activity.lifecycleScope`
</Accordion>

<Accordion title="cleanup()">
  Clean up all resources. Should be called when no longer in use.

  ```kotlin theme={null}
  fun cleanup()
  ```
</Accordion>

***

### Avatar

Avatar data class containing core avatar information.

```kotlin theme={null}
class Avatar
```

| Property     | Type     | Description             |
| ------------ | -------- | ----------------------- |
| `id`         | `String` | The avatar identifier.  |
| `pointCount` | `Int`    | The avatar point count. |

### Configuration

SDK configuration class.

```kotlin theme={null}
data class Configuration
```

##### Initializers

<Accordion title="Configuration(environment, audioFormat, drivingServiceMode, logLevel)">
  Creates a new configuration with the specified parameters.

  ```kotlin theme={null}
  data class Configuration(
      val environment: Environment,
      val audioFormat: AudioFormat = AudioFormat(16000),
      val drivingServiceMode: DrivingServiceMode = DrivingServiceMode.SDK,
      val logLevel: LogLevel = LogLevel.INFO
  )
  ```

  **Parameters:**

  * `environment`: The environment to use
  * `audioFormat`: The audio format configuration
  * `drivingServiceMode`: The driving service mode
  * `logLevel`: The log level
</Accordion>

### Environment

Environment enum.

```kotlin theme={null}
enum class Environment
```

| Case   | Description                |
| ------ | -------------------------- |
| `intl` | International environment. |
| `cn`   | China environment.         |
| `test` | Test environment.          |

### AudioFormat

Audio format configuration for AvatarKit.

```kotlin theme={null}
class AudioFormat
```

##### Initializers

<Accordion title="AudioFormat(sampleRate:)">
  Creates a new audio format with the specified sample rate.

  ```kotlin theme={null}
  class AudioFormat(val sampleRate: Int)
  ```

  **Parameters:**

  * `sampleRate`: The audio sample rate in Hz. Supported sample rates are: 8000, 16000, 22050, 24000, 32000, 44100, 48000.
</Accordion>

### DrivingServiceMode

Driving service modes for AvatarKit.

```kotlin theme={null}
enum class DrivingServiceMode
```

| Case   | Description                                   |
| ------ | --------------------------------------------- |
| `SDK`  | The SDK handles driving service internally.   |
| `HOST` | The host application handles driving service. |

### LogLevel

Log levels for AvatarKit.

```kotlin theme={null}
enum class LogLevel
```

| Case      | Description           |
| --------- | --------------------- |
| `ALL`     | Log all messages.     |
| `VERBOSE` | Log verbose messages. |
| `DEBUG`   | Log debug messages.   |
| `INFO`    | Log info messages.    |
| `WARNING` | Log warning messages. |
| `ERROR`   | Log error messages.   |
| `OFF`     | Disable logging.      |

### ConnectionState

Connection state sealed class.

```kotlin theme={null}
sealed class ConnectionState
```

| Case                | Description                              |
| ------------------- | ---------------------------------------- |
| `Connecting`        | The connection is being established.     |
| `Connected`         | The connection is active.                |
| `Disconnected`      | The connection has been closed.          |
| `Failed(Exception)` | The connection failed with an exception. |

### ConversationState

Avatar conversation state enum.

```kotlin theme={null}
enum class ConversationState
```

| Case      | Description                              |
| --------- | ---------------------------------------- |
| `Idle`    | Idle state, showing breathing animation. |
| `Active`  | Active, waiting for playable content.    |
| `Playing` | The avatar is playing audio/animation.   |

### LoadProgress

Load progress sealed class.

```kotlin theme={null}
sealed class LoadProgress
```

| Case                 | Description                      |
| -------------------- | -------------------------------- |
| `Downloading(Float)` | Downloading with progress (0-1). |
| `Completed`          | Loading completed.               |
| `Failed(Throwable)`  | Loading failed with error.       |


# Golang SDK
Source: https://docs.spatialreal.ai/sdk-reference/go-sdk/go-sdk

Using the Golang SDK

## Repository

The Golang SDK is available on GitHub: [spatialwalk/avatar-sdk-go](https://github.com/spatialwalk/avatar-sdk-go).

```bash theme={null}
go get github.com/spatialwalk/avatar-sdk-go
```

## Quick start

```go theme={null}
import (
	avatarsdkgo "github.com/spatialwalk/avatar-sdk-go"
)

func main() {
	session := avatarsdkgo.NewAvatarSession(
		avatarsdkgo.WithAPIKey("your-api-key"),
		avatarsdkgo.WithAppID("your-app-id"),
		avatarsdkgo.WithAvatarID("your-avatar-id"),
		avatarsdkgo.WithConsoleEndpointURL("https://console.us-west.spatialwalk.cloud/v1/console"),
		avatarsdkgo.WithIngressEndpointURL("wss://api.us-west.spatialwalk.cloud/v2/driveningress"),
		avatarsdkgo.WithExpireAt(time.Now().Add(5 * time.Minute)),
		avatarsdkgo.WithTransportFrames(func(data []byte, last bool) {
			// handle animation frame
		}),
		avatarsdkgo.WithOnError(func(err error) {
			// handle error
		}),
		avatarsdkgo.WithOnClose(func() {
			// handle close
		}),
	)

	if err := session.Init(ctx); err != nil {
		// handle init error
	}

	connectionID, err := session.Start(ctx)
	if err != nil {
		// handle start error
	}
	defer session.Close()

	// Send audio
	requestID, err := session.SendAudio(audioData, true)
	if err != nil {
		// handle error
	}
}
```

## Egress Mode

For real-time applications, you can configure egress to stream avatar output directly to a real-time platform like [**LiveKit** (rooms)](https://docs.livekit.io/intro/overview/). See [LiveKit Agent server side guide](/guide/rtc-livekit-server) for full configuration and code examples.

Quick example with LiveKit:

```go theme={null}
session := avatarsdkgo.NewAvatarSession(
	// ... other options ...
	avatarsdkgo.WithLiveKitEgress(&avatarsdkgo.LiveKitEgressConfig{
		URL:         "wss://your-livekit-server.com",
		APIKey:      "livekit-api-key",
		APISecret:   "livekit-api-secret",
		RoomName:    "your-room-name",
		PublisherID: "avatar-publisher",
	}),
)
```

## Changelog

[https://github.com/spatialwalk/avatar-sdk-go/releases](https://github.com/spatialwalk/avatar-sdk-go/releases)


# API Reference
Source: https://docs.spatialreal.ai/sdk-reference/ios-sdk/api-reference

Browse the complete API reference

### AvatarSDK

Main initialization and configuration interface for AvatarKit.

```swift theme={null}
@MainActor enum AvatarSDK
```

##### Type Properties

<Accordion title="appID">
  The app identifier.

  ```swift theme={null}
  static var appID: String { get }
  ```
</Accordion>

<Accordion title="configuration">
  The app configuration.

  ```swift theme={null}
  static var configuration: Configuration { get }
  ```
</Accordion>

<Accordion title="sessionToken">
  The session token used to authenticate avatars with the AvatarKit Server.

  ```swift theme={null}
  static var sessionToken: String
  ```
</Accordion>

<Accordion title="userID">
  The user identifier provided by the developer to identify the user in the AvatarKit Log Service.

  ```swift theme={null}
  static var userID: String
  ```
</Accordion>

<Accordion title="supportsCurrentDevice">
  Returns a Boolean value that indicates whether AvatarKit supports the current device.

  ```swift theme={null}
  static var supportsCurrentDevice: Bool { get }
  ```
</Accordion>

<Accordion title="version">
  Returns the version of AvatarKit.

  ```swift theme={null}
  static var version: String { get }
  ```
</Accordion>

##### Type Methods

<Accordion title="initialize(appID:configuration:)">
  Initialize AvatarKit.

  ```swift theme={null}
  static func initialize(appID: String, configuration: Configuration)
  ```

  **Parameters:**

  * `appID`: The app identifier
  * `configuration`: The configuration for AvatarKit
</Accordion>

<Accordion title="benchmark()">
  Measures the device's computational performance for avatar rendering.

  ```swift theme={null}
  static func benchmark() async -> Int
  ```

  **Returns:** A performance score indicating the device's capability.
</Accordion>

### AvatarManager

Manage avatar asset loading, caching, and retrieval.

```swift theme={null}
final class AvatarManager
```

##### Type Properties

<Accordion title="shared">
  The shared avatar manager instance.

  ```swift theme={null}
  static let shared: AvatarManager
  ```
</Accordion>

##### Instance Methods

<Accordion title="load(id:onProgress:)">
  Loads an avatar by ID.

  ```swift theme={null}
  func load(id: String, onProgress: (ProgressHandler)?) async throws -> Avatar
  ```

  **Parameters:**

  * `id`: The avatar identifier
  * `onProgress`: Optional progress callback

  **Returns:** The loaded `Avatar` instance.
</Accordion>

<Accordion title="cancelLoading(id:)">
  Cancels the loading of avatar by ID.

  ```swift theme={null}
  func cancelLoading(id: String) async
  ```

  **Parameters:**

  * `id`: The avatar identifier
</Accordion>

<Accordion title="cancelAllLoading()">
  Cancels all loading of avatars.

  ```swift theme={null}
  func cancelAllLoading() async
  ```
</Accordion>

<Accordion title="retrieve(id:)">
  Retrieves a cached avatar by ID.

  ```swift theme={null}
  func retrieve(id: String) -> Avatar?
  ```

  **Parameters:**

  * `id`: The avatar identifier

  **Returns:** The cached `Avatar` if available, otherwise `nil`.
</Accordion>

<Accordion title="derive(assetPath:)">
  Derives an avatar from a local asset path.

  ```swift theme={null}
  func derive(assetPath: String) throws -> Avatar
  ```

  **Parameters:**

  * `assetPath`: The path to the avatar asset

  **Returns:** The derived `Avatar` instance.
</Accordion>

<Accordion title="clear(id:)">
  Clears cached data for a specific avatar.

  ```swift theme={null}
  func clear(id: String) throws
  ```

  **Parameters:**

  * `id`: The avatar identifier to clear
</Accordion>

<Accordion title="clearAll()">
  Clears all cached avatar data.

  ```swift theme={null}
  func clearAll() throws
  ```
</Accordion>

<Accordion title="clearLRU(keepCount:)">
  Clears least recently used avatars, keeping the specified count.

  ```swift theme={null}
  func clearLRU(keepCount: Int) throws
  ```

  **Parameters:**

  * `keepCount`: Number of most recently used avatars to keep
</Accordion>

<Accordion title="getCacheSize(id:)">
  Gets the cache size for a specific avatar.

  ```swift theme={null}
  func getCacheSize(id: String) throws -> Int
  ```

  **Parameters:**

  * `id`: The avatar identifier

  **Returns:** The cache size in bytes.
</Accordion>

<Accordion title="getAllCacheSize()">
  Gets the total cache size for all avatars.

  ```swift theme={null}
  func getAllCacheSize() throws -> Int
  ```

  **Returns:** The total cache size in bytes.
</Accordion>

### AvatarController

The main controller for managing avatar driving service connections and interactions.

```swift theme={null}
@MainActor final class AvatarController
```

##### Instance Properties

<Accordion title="onFirstRendering">
  Callback when the avatar is first rendered.

  ```swift theme={null}
  var onFirstRendering: (() -> Void)?
  ```
</Accordion>

<Accordion title="onConnectionState">
  Callback for connection state changes.

  ```swift theme={null}
  var onConnectionState: ((ConnectionState) -> Void)?
  ```
</Accordion>

<Accordion title="onConversationState">
  Callback for conversation state changes.

  ```swift theme={null}
  var onConversationState: ((ConversationState) -> Void)?
  ```
</Accordion>

<Accordion title="onError">
  Callback for error events.

  ```swift theme={null}
  var onError: ((AvatarError) -> Void)?
  ```
</Accordion>

<Accordion title="start()">
  Starts the avatar driving service connection.

  ```swift theme={null}
  func start()
  ```
</Accordion>

<Accordion title="close()">
  Closes the avatar driving service.

  ```swift theme={null}
  func close()
  ```
</Accordion>

<Accordion title="pauseRendering()">
  Pauses avatar rendering.

  ```swift theme={null}
  func pauseRendering()
  ```
</Accordion>

<Accordion title="resumeRendering()">
  Resumes avatar rendering.

  ```swift theme={null}
  func resumeRendering()
  ```
</Accordion>

<Accordion title="isRendering">
  Whether the current avatar is actually rendering.

  ```swift theme={null}
  var isRendering: Bool { get }
  ```
</Accordion>

<Accordion title="interrupt()">
  Stops playback and terminates the current conversation.

  ```swift theme={null}
  func interrupt()
  ```
</Accordion>

<Accordion title="send(_:end:)">
  Sends audio to the avatar driving service.

  ```swift theme={null}
  func send(_ data: Data, end: Bool) -> String
  ```

  **Parameters:**

  * `data`: The audio data to send
  * `end`: Whether this is the end of the audio stream

  **Returns:** A conversation ID string.
</Accordion>

<Accordion title="yield(_:end:audioFormat:)">
  Yields audio from the host.

  ```swift theme={null}
  func yield(_ data: Data, end: Bool, audioFormat: AudioFormat?) -> String
  ```

  **Parameters:**

  * `data`: The audio data
  * `end`: Whether this is the end of the audio stream
  * `audioFormat`: The audio format (optional)

  **Returns:** A conversation ID string.
</Accordion>

<Accordion title="yield(_:conversationID:)">
  Yields animations from the host.

  ```swift theme={null}
  func yield(_ animations: [Data], conversationID: String)
  ```

  **Parameters:**

  * `animations`: Array of animation data
  * `conversationID`: The conversation identifier
</Accordion>

<Accordion title="pointCount">
  The point count of the current avatar.

  ```swift theme={null}
  var pointCount: Int { get }
  ```
</Accordion>

<Accordion title="volume">
  The volume of playback.

  ```swift theme={null}
  var volume: Float
  ```
</Accordion>

### AvatarView

A UIView subclass for rendering avatars.

```swift theme={null}
@MainActor final class AvatarView: UIView
```

##### Initializers

<Accordion title="init(avatar:)">
  Creates a new avatar view with the specified avatar.

  ```swift theme={null}
  init(avatar: Avatar)
  ```

  **Parameters:**

  * `avatar`: The avatar to display
</Accordion>

##### Instance Properties

<Accordion title="controller">
  The controller for the avatar.

  ```swift theme={null}
  var controller: AvatarController { get }
  ```
</Accordion>

<Accordion title="contentTransform">
  Transform for avatar content position and scale within the view.

  ```swift theme={null}
  var contentTransform: Transform
  ```
</Accordion>

<Accordion title="isOpaque">
  Whether the view is opaque.

  ```swift theme={null}
  var isOpaque: Bool
  ```
</Accordion>

***

### Avatar

Represents an avatar instance.

```swift theme={null}
struct Avatar
```

| Instance Property | Type     | Description                                   |
| ----------------- | -------- | --------------------------------------------- |
| `id`              | `String` | The avatar identifier.                        |
| `isFromCache`     | `Bool`   | Whether the instance of avatar is from cache. |

### Configuration

Configuration for AvatarKit.

```swift theme={null}
struct Configuration
```

##### Initializers

<Accordion title="init(environment:audioFormat:drivingServiceMode:logLevel:)">
  Creates a new configuration with the specified parameters.

  ```swift theme={null}
  init(
      environment: Environment,
      audioFormat: AudioFormat,
      drivingServiceMode: DrivingServiceMode,
      logLevel: LogLevel
  )
  ```

  **Parameters:**

  * `environment`: The environment to use
  * `audioFormat`: The audio format configuration
  * `drivingServiceMode`: The driving service mode
  * `logLevel`: The log level
</Accordion>

| Instance Property    | Type                 | Description                         |
| -------------------- | -------------------- | ----------------------------------- |
| `environment`        | `Environment`        | Environment for AvatarKit.          |
| `audioFormat`        | `AudioFormat`        | Audio format for AvatarKit.         |
| `drivingServiceMode` | `DrivingServiceMode` | Driving service mode for AvatarKit. |
| `logLevel`           | `LogLevel`           | Log level for AvatarKit.            |

### Environment

Environments for AvatarKit.

```swift theme={null}
enum Environment
```

| Case   | Description                |
| ------ | -------------------------- |
| `intl` | International environment. |
| `cn`   | China environment.         |

### AudioFormat

Audio format configuration for AvatarKit.

```swift theme={null}
struct AudioFormat
```

##### Initializers

<Accordion title="init(sampleRate:)">
  Creates a new audio format with the specified sample rate.

  ```swift theme={null}
  init(sampleRate: Int)
  ```

  **Parameters:**

  * `sampleRate`: The audio sample rate in Hz
</Accordion>

| Instance Property | Type  | Description                                        |
| ----------------- | ----- | -------------------------------------------------- |
| `sampleRate`      | `Int` | The audio sample rate in Hz.                       |
| `channelCount`    | `Int` | The number of audio channels. Fixed to 1 for mono. |

### DrivingServiceMode

Driving service modes for AvatarKit.

```swift theme={null}
enum DrivingServiceMode
```

| Case   | Description                                   |
| ------ | --------------------------------------------- |
| `sdk`  | The SDK handles driving service internally.   |
| `host` | The host application handles driving service. |

### LogLevel

Log levels for AvatarKit.

```swift theme={null}
enum LogLevel
```

| Case      | Description                   |
| --------- | ----------------------------- |
| `all`     | Log all messages.             |
| `warning` | Log warnings and errors only. |
| `error`   | Log errors only.              |
| `off`     | Disable logging.              |

### ConnectionState

Connection states for AvatarKit.

```swift theme={null}
enum ConnectionState
```

| Case            | Description                          |
| --------------- | ------------------------------------ |
| `connecting`    | The connection is being established. |
| `connected`     | The connection is active.            |
| `disconnected`  | The connection has been closed.      |
| `failed(Error)` | The connection failed with an error. |

### ConversationState

Conversation states for AvatarKit.

```swift theme={null}
enum ConversationState
```

| Case      | Description                            |
| --------- | -------------------------------------- |
| `idle`    | No active conversation.                |
| `playing` | The avatar is playing audio/animation. |
| `paused`  | The conversation is paused.            |

### Transform

Transform for avatar content rendering.

```swift theme={null}
struct Transform
```

##### Initializers

<Accordion title="init(x:y:scale:)">
  Creates a new transform with the specified translation and scale.

  ```swift theme={null}
  init(x: Float, y: Float, scale: Float)
  ```

  **Parameters:**

  * `x`: The x-axis translation
  * `y`: The y-axis translation
  * `scale`: The scale factor
</Accordion>

| Type Property | Type        | Description             |
| ------------- | ----------- | ----------------------- |
| `identity`    | `Transform` | The identity transform. |

| Instance Property | Type    | Description             |
| ----------------- | ------- | ----------------------- |
| `x`               | `Float` | The x-axis translation. |
| `y`               | `Float` | The y-axis translation. |
| `scale`           | `Float` | The scale factor.       |

### AvatarError

Avatar errors for AvatarKit.

```swift theme={null}
enum AvatarError: LocalizedError
```

| Case                           | Description                       |
| ------------------------------ | --------------------------------- |
| `appIDUnrecognized`            | The app ID is not recognized.     |
| `avatarIDUnrecognized`         | The avatar ID is not recognized.  |
| `avatarAssetMissing`           | The avatar asset is missing.      |
| `failedToDownloadAvatarAssets` | Failed to download avatar assets. |
| `failedToFetchAvatarMetadata`  | Failed to fetch avatar metadata.  |
| `sessionTokenExpired`          | The session token has expired.    |
| `sessionTokenInvalid`          | The session token is invalid.     |

| Instance Property  | Type      | Description                           |
| ------------------ | --------- | ------------------------------------- |
| `errorDescription` | `String?` | A localized description of the error. |
| `failureReason`    | `String?` | The reason for the failure.           |


# JavaScript SDK
Source: https://docs.spatialreal.ai/sdk-reference/js-sdk/js-sdk

Server SDK for JavaScript/Node.js

Coming soon.


# Python SDK
Source: https://docs.spatialreal.ai/sdk-reference/python-sdk/python-sdk

Using the Python Server SDK

A Python SDK for connecting to avatar services via WebSocket, supporting audio streaming and receiving animation frames.

## Repository

The Python SDK is available on GitHub: [spatialwalk/avatar-sdk-python](https://github.com/spatialwalk/avatar-sdk-python) and [PyPI](https://pypi.org/project/avatarkit/).

```bash theme={null}
pip install avatarkit
```

## Requirements

* `api_key`
* `app_id`
* `avatar_id`
* console endpoint URL
* ingress endpoint URL

## Endpoints

Build your endpoint URLs using your selected region (see [Regions](/api-reference/regions)):

* Console endpoint URL: `https://console.<region>.spatialwalk.cloud/v1/console`
* Ingress endpoint URL: `wss://api.<region>.spatialwalk.cloud/v2/driveningress`

## Quick Start

```python theme={null}
import asyncio
from datetime import datetime, timedelta, timezone

from avatarkit import new_avatar_session


async def main():
    session = new_avatar_session(
        api_key="your-api-key",
        app_id="your-app-id",
        console_endpoint_url="https://console.us-west.spatialwalk.cloud/v1/console",
        ingress_endpoint_url="wss://api.us-west.spatialwalk.cloud/v2/driveningress",
        avatar_id="your-avatar-id",
        expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
        transport_frames=lambda frame, last: print(f"Received frame: {len(frame)} bytes"),
        on_error=lambda err: print(f"Error: {err}"),
        on_close=lambda: print("Session closed"),
    )

    await session.init()
    connection_id = await session.start()
    print(f"Connected: {connection_id}")

    audio_data = b"..."  # Your PCM audio data
    request_id = await session.send_audio(audio_data, end=True)
    print(f"Sent audio: {request_id}")

    await asyncio.sleep(10)
    await session.close()


if __name__ == "__main__":
    asyncio.run(main())
```

## Session Configuration

Use `new_avatar_session()` to configure and create a session:

```python theme={null}
from datetime import datetime, timedelta, timezone

from avatarkit import new_avatar_session


session = new_avatar_session(
    avatar_id="avatar-123",
    api_key="your-api-key",
    app_id="your-app-id",
    # Set use_query_auth=True for web-style auth that sends appId and sessionKey
    # in the WebSocket query string instead of request headers.
    use_query_auth=False,
    expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
    console_endpoint_url="https://console.us-west.spatialwalk.cloud/v1/console",
    ingress_endpoint_url="wss://api.us-west.spatialwalk.cloud/v2/driveningress",
    sample_rate=16000,
    transport_frames=on_frame_received,
    on_error=on_error,
    on_close=on_close,
)
```

## Session Lifecycle

```python theme={null}
# 1. Initialize and request a session token
await session.init()

# 2. Start the WebSocket connection
connection_id = await session.start()

# 3. Send audio data
request_id = await session.send_audio(audio_bytes, end=True)

# 4. Receive animation frames through the callback

# 5. Close the session
await session.close()
```

## Audio Format

The SDK currently supports **mono 16-bit PCM (`s16le`)** audio:

* Sample rate: `8000`, `16000`, `22050`, `24000`, `32000`, `44100`, `48000`
* Channels: `1` (mono)
* Bit depth: `16-bit`
* Format: raw PCM bytes

```python theme={null}
with open("audio.pcm", "rb") as f:
    audio_data = f.read()

await session.send_audio(audio_data, end=True)
```

## LiveKit Egress Mode

When configured with `livekit_egress`, audio and animation data are streamed to a LiveKit room instead of being returned through the WebSocket connection. For end-to-end integration details, see the [LiveKit Agent server side guide](/guide/rtc-livekit-server).

```python theme={null}
from datetime import datetime, timedelta, timezone

from avatarkit import LiveKitEgressConfig, new_avatar_session


session = new_avatar_session(
    avatar_id="avatar-123",
    api_key="your-api-key",
    app_id="your-app-id",
    console_endpoint_url="https://console.us-west.spatialwalk.cloud/v1/console",
    ingress_endpoint_url="wss://api.us-west.spatialwalk.cloud/v2/driveningress",
    expire_at=datetime.now(timezone.utc) + timedelta(minutes=5),
    livekit_egress=LiveKitEgressConfig(
        url="wss://livekit.example.com",
        api_key="livekit-api-key",
        api_secret="livekit-api-secret",
        room_name="my-room",
        publisher_id="avatar-publisher",
    ),
)
```

When LiveKit egress is enabled:

* the server streams output to the specified LiveKit room
* the `transport_frames` callback is not invoked
* audio and animation data are published under the configured publisher ID

### Interrupt

The `interrupt()` method sends an interrupt signal to stop current audio processing. This is only available when using LiveKit egress mode.

```python theme={null}
request_id = await session.send_audio(audio_data, end=True)

# Later, interrupt the most recent in-flight request.
interrupted_id = await session.interrupt()
print(f"Interrupted request: {interrupted_id}")
```

The interrupt uses the most recent request ID, even after `end=True` has been sent.

## Callbacks

### `transport_frames`

Receives animation frames from the server:

```python theme={null}
def on_frame_received(frame_data: bytes, is_last: bool):
    print(f"Received frame: {len(frame_data)} bytes")
    if is_last:
        print("This is the last frame")
```

### `on_error`

Handles structured SDK errors from the session:

```python theme={null}
from avatarkit import AvatarSDKError


def on_error(error: Exception):
    print(f"Session error: {error}")

    if isinstance(error, AvatarSDKError):
        print("  code:", error.code.value)
        print("  phase:", error.phase)
        print("  http_status:", error.http_status)
        print("  server_code:", error.server_code)
        print("  server_detail:", error.server_detail)
```

The SDK reports structured `AvatarSDKError` instances for token creation failures, WebSocket upgrade rejections, handshake failures, runtime `ServerError` messages, and unexpected connection drops.

### `on_close`

Called when the session closes:

```python theme={null}
def on_close():
    print("Session has been closed")
```

## Error Handling

Use `SessionTokenError` for token creation failures and `AvatarSDKError` for all other structured SDK errors:

```python theme={null}
from avatarkit import AvatarSDKError, SessionTokenError


try:
    await session.init()
    await session.start()
except SessionTokenError as error:
    print("token failed", error.code.value, error.server_detail)
except AvatarSDKError as error:
    print("sdk error", error.code.value, error.phase, error.server_detail)
```

`AvatarSDKError` and `SessionTokenError` expose these fields:

* `code` - stable SDK error code
* `message` - human-readable message
* `phase` - failure phase such as `session_token`, `websocket_connect`, `websocket_handshake`, `websocket_runtime`, or `websocket_send`
* `http_status` - HTTP status for token or WebSocket upgrade failures
* `server_code` - server-provided error code, including runtime protobuf `ServerError.code`
* `server_title` / `server_detail` - parsed server error details when available
* `connection_id` / `req_id` - server correlation identifiers when available
* `raw_body` - raw HTTP rejection body for token or WebSocket upgrade failures
* `close_code` / `close_reason` - WebSocket close details for unexpected disconnects

### Common `AvatarSDKErrorCode` values

* `sessionTokenExpired` - session token expired or unauthorized
* `sessionTokenInvalid` - invalid or empty session token
* `appIDUnrecognized` - App ID is not recognized by the server
* `appIDMismatch` - session token belongs to a different app
* `avatarNotFound` - avatar does not exist
* `billingRequired` - session denied by billing checks
* `creditsExhausted` - runtime or connect-time credits exhausted
* `sessionDurationExceeded` - billing-enforced session timeout reached
* `unsupportedSampleRate` - handshake rejected unsupported audio sample rate
* `invalidEgressConfig` - LiveKit or Agora egress config is invalid
* `egressUnavailable` - egress service is unavailable or not configured
* `idleTimeout` - server closed the session after input inactivity
* `upstreamError` - internal upstream service failed
* `protocolError` - invalid protobuf or unexpected message sequence
* `connectionFailed` - transport-level connection failure
* `connectionClosed` - unexpected WebSocket close
* `serverError` - server-side failure that did not match a more specific mapping
* `invalidRequest` - other client-side request validation errors
* `unknown` - fallback when the SDK cannot classify the failure

For a cross-SDK overview, see [Error Codes](/resources/error-codes).

## API Reference

### `AvatarSession`

Main class for managing avatar sessions.

#### Methods

* `async init()` - initialize the session and obtain a token
* `async start() -> str` - start the WebSocket connection and return a `connection_id`
* `async send_audio(audio: bytes, end: bool = False) -> str` - send audio data and return a request ID
* `async interrupt() -> str` - interrupt current audio processing in LiveKit egress mode
* `async close()` - close the session and clean up resources
* `config -> SessionConfig` - current session configuration

### `SessionConfig`

Configuration dataclass for avatar sessions.

#### Fields

* `avatar_id: str` - avatar identifier
* `api_key: str` - API key for authentication
* `app_id: str` - application identifier
* `use_query_auth: bool` - send WebSocket auth in query params instead of headers
* `expire_at: datetime` - session expiration time
* `sample_rate: int` - audio sample rate, default `16000`
* `bitrate: int` - audio bitrate, default `0`
* `transport_frames: Callable[[bytes, bool], None]` - frame callback
* `on_error: Callable[[Exception], None]` - error callback
* `on_close: Callable[[], None]` - close callback
* `console_endpoint_url: str` - Console API URL
* `ingress_endpoint_url: str` - ingress WebSocket URL
* `livekit_egress: Optional[LiveKitEgressConfig]` - LiveKit egress configuration

### `LiveKitEgressConfig`

Configuration for streaming to a LiveKit room.

#### Fields

* `url: str` - LiveKit server URL, for example `wss://livekit.example.com`
* `api_key: str` - LiveKit API key
* `api_secret: str` - LiveKit API secret
* `room_name: str` - room name to join
* `publisher_id: str` - publisher identity in the room
* `extra_attributes: dict[str, str]` - extra participant attributes
* `idle_timeout: int` - idle timeout in seconds, `0` uses server defaults

### Utility Functions

* `generate_log_id() -> str` - generate a unique log ID in the format `YYYYMMDDHHMMSS_<nanoid>`

### Exceptions

* `AvatarSDKError` - structured SDK error with stable code and context fields
* `SessionTokenError` - subclass of `AvatarSDKError` raised when session token creation fails

## Changelog

[avatar-sdk-python releases](https://github.com/spatialwalk/avatar-sdk-python/releases)


# API Reference
Source: https://docs.spatialreal.ai/sdk-reference/web-sdk/api-reference

Complete API documentation for AvatarKit Web SDK

## Quick Reference

| Class              | Purpose            | Key Methods                                   |
| ------------------ | ------------------ | --------------------------------------------- |
| `AvatarSDK`        | SDK initialization | `initialize()`, `setSessionToken()`           |
| `AvatarManager`    | Avatar loading     | `load()`                                      |
| `AvatarView`       | 3D rendering       | `onFirstRendering`, `dispose()`               |
| `AvatarController` | Communication      | `start()`, `send()`, `pause()`, `interrupt()` |

<Warning>
  **Important:** App ID and Session Token are paired and must be used together. Each App ID corresponds to a specific list of avatars that can be driven. You must obtain both App ID and Session Token from the [SpatialReal Studio](https://app.spatialreal.ai).
</Warning>

```typescript theme={null}
export class AvatarSDK
```

***

## AvatarSDK

Main entry point for SDK initialization and global configuration.

```typescript theme={null}
import { AvatarSDK, Environment } from '@spatialwalk/avatarkit'
```

### Static Properties

| Property        | Type             | Description                |
| --------------- | ---------------- | -------------------------- |
| `isInitialized` | `boolean`        | Whether SDK is initialized |
| `appId`         | `string \| null` | Current App ID             |
| `configuration` | `Configuration`  | Current SDK configuration  |
| `sessionToken`  | `string \| null` | Current session token      |
| `userId`        | `string \| null` | Current user ID            |
| `version`       | `string`         | SDK version                |

### Static Methods

#### initialize(appId, configuration)

Initialize the SDK. Must be called before any other operations.

```typescript theme={null}
await AvatarSDK.initialize('your-app-id', {
  environment: Environment.cn,  // or Environment.intl
  drivingServiceMode: DrivingServiceMode.sdk,  // Optional, default: sdk
  logLevel: LogLevel.warning,  // Optional, default: off
  audioFormat: {  // Optional
    channelCount: 1,
    sampleRate: 16000,
  },
})
```

#### setSessionToken(token)

Set session token for authentication. **Only required for SDK Mode** (server-driven animation).

```typescript theme={null}
AvatarSDK.setSessionToken('your-session-token')
```

**Important:**

* **Only needed for SDK Mode** - not required for Host Mode or RTC Mode
* Token must be obtained from your backend server
* Token has max 24 hours validity
* Token must be paired with App ID

#### setUserId(userId)

Set user ID for logging and analytics.

```typescript theme={null}
AvatarSDK.setUserId('user-123')
```

#### cleanup()

Release all SDK resources. Call when SDK is no longer needed.

```typescript theme={null}
AvatarSDK.cleanup()
```

***

## AvatarManager

Handles avatar loading and caching.

```typescript theme={null}
import { AvatarManager } from '@spatialwalk/avatarkit'
```

### Static Properties

| Property | Type            | Description        |
| -------- | --------------- | ------------------ |
| `shared` | `AvatarManager` | Singleton instance |

### Instance Methods

#### load(id, onProgress?)

Load an avatar by ID.

```typescript theme={null}
const avatar = await AvatarManager.shared.load('avatar-id', (progress) => {
  switch (progress.type) {
    case 'downloading':
      console.log(`Loading: ${progress.progress}%`)
      break
    case 'completed':
      console.log('Load complete')
      break
    case 'failed':
      console.error('Load failed:', progress.error)
      break
  }
})
```

**Parameters:**

| Parameter    | Type                                   | Description       |
| ------------ | -------------------------------------- | ----------------- |
| `id`         | `string`                               | Avatar ID         |
| `onProgress` | `(progress: LoadProgressInfo) => void` | Progress callback |

**Returns:** `Promise<Avatar | null>`

#### clearAll()

Clear all cached avatar resources.

```typescript theme={null}
AvatarManager.shared.clearAll()
```

***

## AvatarView

3D rendering view that displays the avatar.

```typescript theme={null}
import { AvatarView } from '@spatialwalk/avatarkit'
```

### Constructor

```typescript theme={null}
const avatarView = new AvatarView(avatar, container)
```

| Parameter   | Type          | Description                                     |
| ----------- | ------------- | ----------------------------------------------- |
| `avatar`    | `Avatar`      | Loaded avatar object                            |
| `container` | `HTMLElement` | Container element (canvas auto-fills container) |

<Warning>
  **⚠️ Important:** The container element MUST have non-zero width and height. The canvas will automatically fill the container size.
</Warning>

### Instance Properties

| Property           | Type               | Description                       |
| ------------------ | ------------------ | --------------------------------- |
| `controller`       | `AvatarController` | Communication controller          |
| `onFirstRendering` | `() => void`       | Callback when first frame renders |
| `transform`        | `{ x, y, scale }`  | Avatar position and scale         |

### Instance Methods

#### dispose()

Clean up view resources. Call when view is no longer needed.

```typescript theme={null}
avatarView.dispose()
```

***

## AvatarController

Handles real-time communication with the avatar service.

```typescript theme={null}
// Accessed via AvatarView
const controller = avatarView.controller
```

### Instance Properties

| Property            | Type                | Description                |
| ------------------- | ------------------- | -------------------------- |
| `connectionState`   | `ConnectionState`   | Current connection state   |
| `conversationState` | `ConversationState` | Current conversation state |

### Event Callbacks

```typescript theme={null}
// Connection state changes
controller.onConnectionState = (state: ConnectionState) => {
  // 'disconnected' | 'connecting' | 'connected' | 'failed'
}

// Conversation state changes
controller.onConversationState = (state: ConversationState) => {
  // 'idle' | 'playing' | 'pausing'
}

// Error handling
controller.onError = (error: Error) => {
  console.error('Error:', error)
}
```

### Instance Methods

#### initializeAudioContext()

Initialize audio context. **MUST be called in user gesture context.**

```typescript theme={null}
button.addEventListener('click', async () => {
  await controller.initializeAudioContext()
})
```

#### start()

Connect to the avatar service. SDK mode only.

```typescript theme={null}
await controller.start()
```

#### send(audioData, end)

Send audio data to server. SDK mode only.

```typescript theme={null}
// Send audio data (continue)
controller.send(audioData, false)

// Send final audio data (end conversation)
controller.send(audioData, true)
```

| Parameter   | Type          | Description                     |
| ----------- | ------------- | ------------------------------- |
| `audioData` | `ArrayBuffer` | PCM16 mono audio data           |
| `end`       | `boolean`     | Whether this is the final chunk |

**Returns:** `string` - Conversation ID

#### close()

Close service connection. SDK mode only.

```typescript theme={null}
controller.close()
```

### Host Mode Methods

For Host Mode (`DrivingServiceMode.host`) only:

#### yieldAudioData(audioData, end)

Provide audio data in Host Mode.

```typescript theme={null}
const conversationId = controller.yieldAudioData(audioData, false)
```

#### yieldFramesData(frames, conversationId)

Provide animation frames in Host Mode.

```typescript theme={null}
controller.yieldFramesData(animationDataArray, conversationId)
```

### Common Methods (Both Modes)

#### pause()

Pause current playback.

```typescript theme={null}
controller.pause()
```

#### resume()

Resume paused playback.

```typescript theme={null}
await controller.resume()
```

#### interrupt()

Interrupt current playback and clear data.

```typescript theme={null}
controller.interrupt()
```

#### clear()

Clear all data and resources.

```typescript theme={null}
controller.clear()
```

#### getCurrentConversationId()

Get the current active conversation ID.

```typescript theme={null}
const conversationId = controller.getCurrentConversationId()
// Returns: string | null
```

#### setVolume(volume) / getVolume()

Control playback volume (avatar audio only, not system volume).

```typescript theme={null}
controller.setVolume(0.5)  // 50% volume (0.0 to 1.0)
const currentVolume = controller.getVolume()
```

***

## Types and Enums

### Configuration

```typescript theme={null}
interface Configuration {
  environment: Environment
  drivingServiceMode?: DrivingServiceMode  // Default: 'sdk'
  logLevel?: LogLevel                       // Default: 'off'
  audioFormat?: AudioFormat                 // Default: { channelCount: 1, sampleRate: 16000 }
}
```

### Environment

```typescript theme={null}
enum Environment {
  cn = 'cn',      // China region
  intl = 'intl'   // International region
}
```

### DrivingServiceMode

```typescript theme={null}
enum DrivingServiceMode {
  sdk = 'sdk',    // SDK handles WebSocket communication
  host = 'host'   // Host provides audio and animation data
}
```

### LogLevel

```typescript theme={null}
enum LogLevel {
  off = 'off',
  error = 'error',
  warning = 'warning',
  all = 'all'
}
```

### ConnectionState

```typescript theme={null}
enum ConnectionState {
  disconnected = 'disconnected',
  connecting = 'connecting',
  connected = 'connected',
  failed = 'failed'
}
```

### ConversationState

```typescript theme={null}
enum ConversationState {
  idle = 'idle',        // Breathing animation
  playing = 'playing',  // Active conversation
  pausing = 'pausing'   // Paused during playback
}
```

### LoadProgressInfo

```typescript theme={null}
type LoadProgressInfo = {
  type: 'downloading' | 'completed' | 'failed'
  progress?: number  // 0-100
  error?: Error
}
```

### AudioFormat

```typescript theme={null}
interface AudioFormat {
  channelCount: 1  // Fixed to mono
  sampleRate: number  // 8000 | 16000 | 22050 | 24000 | 32000 | 44100 | 48000
}
```

***

## Error Codes

| Code | Name                        | Description             | Solution                 |
| ---- | --------------------------- | ----------------------- | ------------------------ |
| 1001 | `sdkNotVerified`            | SDK not verified        | Check SDK initialization |
| 1002 | `avatarIdWrong`             | Invalid avatar ID       | Verify avatar ID         |
| 1003 | `avatarAssetsMissing`       | Missing avatar assets   | Check avatar resources   |
| 1004 | `avatarCameraSettingsWrong` | Invalid camera settings | Check camera config      |
| 1005 | `serviceError`              | Server error            | Check server status      |
| 1008 | `sendDataWrong`             | Invalid audio data      | Verify PCM16 format      |
| 1009 | `requestTimeout`            | Request timeout         | Check network            |

***

## Complete Usage Example

```typescript theme={null}
import {
  AvatarSDK,
  AvatarManager,
  AvatarView,
  Environment,
  ConnectionState,
  ConversationState,
} from '@spatialwalk/avatarkit'

class AvatarApp {
  private avatarView: AvatarView | null = null

  async init(appId: string, sessionToken: string, avatarId: string, container: HTMLElement) {
    // Initialize SDK
    await AvatarSDK.initialize(appId, { environment: Environment.cn })
    AvatarSDK.setSessionToken(sessionToken)

    // Load avatar
    const avatar = await AvatarManager.shared.load(avatarId)
    if (!avatar) throw new Error('Failed to load avatar')

    // Create view
    this.avatarView = new AvatarView(avatar, container)
    this.avatarView.onFirstRendering = () => {
      console.log('First frame rendered')
    }

    // Set up handlers
    this.avatarView.controller.onConnectionState = (state) => {
      console.log('Connection:', state)
    }
    this.avatarView.controller.onConversationState = (state) => {
      console.log('Conversation:', state)
    }
    this.avatarView.controller.onError = (error) => {
      console.error('Error:', error)
    }
  }

  // Must be called in user gesture context
  async start() {
    await this.avatarView?.controller.initializeAudioContext()
    await this.avatarView?.controller.start()
  }

  send(audioData: ArrayBuffer, isEnd: boolean) {
    this.avatarView?.controller.send(audioData, isEnd)
  }

  interrupt() {
    this.avatarView?.controller.interrupt()
  }

  dispose() {
    this.avatarView?.controller.close()
    this.avatarView?.dispose()
    this.avatarView = null
  }
}
```


# API Keys
Source: https://docs.spatialreal.ai/studio/aki-key

Create and manage API credentials

API Keys are the foundation of your SpatialReal integration. They allow you to authenticate your applications and securely access SpatialReal services. In this guide, you'll learn how to create API Keys in [SpatialReal Studio](https://app.spatialreal.ai) and best practices for managing them.

## Creating an API Key

<Steps>
  <Step title="Open API Keys page">
    Go to [https://app.spatialreal.ai/apps](https://app.spatialreal.ai/apps).
  </Step>

  <Step title="Click Create API Key">
    On the API key page, click **Create API Key**.

    <Frame>
      <img alt="API Keys page" />
    </Frame>
  </Step>

  <Step title="Enter application name">
    Type your **Application Name** and click **Create**.

    <Frame>
      <img alt="Click Create API Key" />
    </Frame>
  </Step>

  <Step title="Copy App ID and API Key">
    In the popup, copy both **App ID** and **API Key**.

    <Frame>
      <img alt="Create app and view popup" />
    </Frame>
  </Step>
</Steps>

***

## Manage your API Key

Your API Key is used to authenticate server-side requests to SpatialReal services.

<Warning>
  **Important:** Keep your API Key secure and never expose it in client-side code or commit it to version control. Treat it like a password to prevent unauthorized access to your SpatialReal account.
</Warning>

**Best Practices:**

* Store your API Key in environment variables or a secrets manager
* Never expose your API Key in client-side code
* Never commit your API Key to version control

***

## Temporary Session Token

For quick testing and development, you can generate a **Temporary Session Token** directly in [SpatialReal Studio](https://app.spatialreal.ai).

<Note>
  Temporary Session Tokens are valid for **24 hours** only. They are intended for testing purposes and should not be used in production.
</Note>

<Steps>
  <Step title="Open Your Application">
    Navigate to [API keys page in SpatialReal Studio](https://app.spatialreal.ai/apps), then click the details button of your application.

    <Frame>
      <img alt="Token Step 1" />
    </Frame>
  </Step>

  <Step title="Generate Token">
    Click **Generate Temporary Token** to create a new temporary session token.

    <Frame>
      <img alt="Token Step 2" />
    </Frame>
  </Step>

  <Step title="Use for Testing">
    Use this token in your client SDK to test avatar rendering without setting up server-side authentication.
  </Step>
</Steps>

***

## Production Authentication

In production, you should generate session tokens from your backend server using the API Key. See the [Authentication Guide](/server/auth) for implementation details.

<Card title="Authentication Guide" icon="lock" href="/server/auth">
  Learn how to implement secure server-side authentication
</Card>


# Public Avatar
Source: https://docs.spatialreal.ai/studio/public-avatar

Browse and use pre-built avatars from the SpatialReal library

## What are Public Avatars?

Public Avatars are a collection of **ready-to-use, high-quality stock avatars** provided by SpatialReal. These avatars are professionally designed and optimized for immediate integration into your applications — no creation or customization needed.

<Frame>
  <img alt="avatars" />
</Frame>

## Accessing Public Avatars

Navigate to the **Public Avatar** page in [SpatialReal Studio](https://app.spatialreal.ai/avatars/library) to browse the complete library of available avatars.

## Using a Public Avatar

<Steps>
  <Step title="Browse the Gallery">
    Explore all available stock avatars displayed as avatar cards in the gallery.
  </Step>

  <Step title="Copy Avatar ID">
    Hover over an avatar card and click the **copy icon** in the **top-right corner** to copy the avatar's unique ID to your clipboard.
  </Step>

  <Step title="Integrate into Your App">
    Use the copied avatar ID in your application code to render the avatar. You can use [guide](/guide/sdk-mode) to see how to do this.
  </Step>
</Steps>

## Next Steps

<CardGroup>
  <Card title="SDK Integration" icon="code" href="/guide/introduction">
    Learn how to integrate avatars into your app
  </Card>
</CardGroup>