What is RTC Mode?
RTC Mode enables real-time voice communication with avatars through WebRTC providers (LiveKit or Agora). It builds on top of Host Mode — the @spatialwalk/avatarkit-rtc adapter package handles the RTC connection and feeds audio/animation data to the SDK automatically.
Web Only: RTC Mode is currently available for Web applications only. iOS and Android support is planned for a future release. In the meantime, native mobile platforms can use Host Mode with your own RTC implementation. When to Use
- Real-time voice conversation — users talk to an avatar via microphone
- Low-latency interaction — WebRTC provides sub-second latency
- Server-side AI — your RTC server processes audio and generates responses
Packages Required
| Package | Purpose | Required |
|---|
@spatialwalk/avatarkit | Avatar rendering SDK | Yes |
@spatialwalk/avatarkit-rtc | RTC adapter | Yes |
[email protected] | LiveKit RTC SDK | Choose one |
agora-rtc-sdk-ng | Agora RTC SDK | Choose one |
Critical: livekit-client must be exactly version 2.16.1. Other versions are not compatible.
LiveKit vs Agora
| Feature | LiveKit | Agora |
|---|
| Protocol | VP8 + RTCRtpScriptTransform | H.264 + SEI |
| Chrome | 94+ | 74+ |
| Firefox | 117+ | 78+ |
| Safari | 15.4+ | 14.1+ |
| Edge | 94+ | 79+ |
| Connection Config | URL + Token + Room Name | App ID + Channel + Token + UID |
| Debug Options | Via logLevel | debugLogging option |
How It Works
RTC Mode uses the SDK in Host Mode internally. The AvatarPlayer acts as a bridge:
- Initializes the avatar SDK with
DrivingServiceMode.host
- Connects to the RTC server via the chosen provider
- Publishes your microphone audio to the RTC server
- Receives animation and audio data from the RTC server
- Feeds animation data into the avatar SDK for rendering; audio is played through the native WebRTC audio track by the RTC provider
You don’t need to call yieldAudioData() or yieldFramesData() manually — the adapter handles this.
What RTC Mode Does NOT Use
Although RTC Mode builds on Host Mode internally, the following SDK/Host Mode features are not used in RTC Mode:
| Feature | Used in SDK/Host Mode | Used in RTC Mode |
|---|
initializeAudioContext() | Yes — required for Web Audio API playback | No — audio is played via WebRTC tracks |
| Internal audio player | Yes — SDK decodes and plays audio internally | No — RTC provider handles audio playback |
yieldAudioData() / yieldFramesData() | Yes — you call these manually | No — the adapter calls them internally |
start() / send() / close() | Yes (SDK Mode only) | No |
Audio path difference: In SDK Mode and Host Mode, the SDK plays avatar audio through its internal audio player (Web Audio API). In RTC Mode, avatar audio arrives as a native WebRTC audio track and is played by the browser’s WebRTC stack directly — the SDK’s internal audio player is not involved. This distinction matters for audio processing (e.g., echo cancellation, noise suppression).
Server-Side Setup
Your backend is responsible for sending audio to the avatar service and having the resulting avatar stream published to your RTC room or channel. Two approaches are available per platform:
| Platform | Framework plugin | Server SDK + egress |
|---|
| LiveKit | LiveKit Agents plugin — hooks into your agent pipeline and publishes to the room | LiveKit Server (section 2) — use AvatarKit Server SDK with LiveKit egress config |
| Agora | Agora TEN plugin — coming soon | Agora Server (section 2) — use AvatarKit Server SDK with Agora egress config |
- Framework plugin: Best if you already use LiveKit Agents or (when available) Agora TEN. The plugin handles audio → Spatialreal → RTC publish for you.
- Server SDK + egress: Use the Golang or Python Server SDK, create a session with LiveKit or Agora egress config, and send audio. The avatar service publishes audio and animation directly to the RTC room/channel; your server does not relay that data.
Client-side setup is the same either way: the client joins the room/channel and uses @spatialwalk/avatarkit-rtc to render the avatar. See the client guides below.
Get Started
Client (browser)
Server (backend)
