Reference integration: real customer-shape code

The 2wai team ships several open-source reference apps that mirror what a real customer integration looks like. This guide is a tour of the architecture every one of them uses on Transport 0 (LiveKit), so you can copy the shape directly or lift pieces into your own build.

Where to read the real code

Every snippet below has a working reference in 2waiorg/2wai-toolkit-public. Pick your language:

• Flutter: apps/avatar_only_flutter — avatar + customer-held brain, ~300 LOC.
• Web (Next.js): samples/nextjs — vanilla React, Route Handlers for mint + brain.
• Node mint proxy: apps/mint_proxy_examples/node — the minimal server piece.
• iOS native (Swift): in preview at ios-native/Samples/HelloAvatar.

Split-plane architecture

Every integration follows the same shape:

                 YOUR backend (mint proxy)           2wai gateway
  client   ────────────────────────────────────────────────────────────
     │  POST /mint (same-origin)     │                      │
     │ ─────────────────────────────▶│  POST /v1/sessions   │
     │                               │  (Bearer sk_, server)│
     │                               │ ────────────────────▶│
     │ ◀── livekit_url,              │◀──── session fields  │
     │     participant_token         │                      │
     │                                                      │
     │  LiveKit room join (participant_token) ─────────────▶│
     │ ◀══ agent participant: video + audio tracks live ════│
     │                                                      │
     │  POST /say {text}             │  your LLM → reply    │
     │ ─────────────────────────────▶│  POST /v1/sessions/  │
     │                               │   {id}/speak (sk_)   │
     │ ◀── {ok}                      │ ────────────────────▶│
     │        …audio/video arrive on the LiveKit room ──────│

Two credentials, scoped very differently:

• The secret key (sk_…) mints sessions, drives speech, and tears them down. It lives only on your backend, in an environment variable. It must never reach a client bundle.
• The participant token is minted per session by the gateway. The client gets only this, and it can do exactly one thing: join the LiveKit room that streams this one session. It can’t mint new sessions or spend money.

Your client talks only to your own backend (same-origin) + one LiveKit room (WebRTC). Your backend makes every sk_-authed call.

Your backend: three routes

The reference backend exposes three routes to the client. This is the complete server-side surface of a minimal integration:

Node (Next.js)

app/api/mint/route.ts (excerpt)

export async function POST(req) {
  const r = await fetch('https://api.2wai.ai/v1/sessions', {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.TWAI_SECRET_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      avatar_id: process.env.AVATAR_ID,
      voice_id: process.env.VOICE_ID,
      mode: 'video',
      transport: 'livekit',
    }),
  });
  const s = await r.json();
  return Response.json({
    session_id: s.session_id,
    livekit_url: s.livekit_url,
    participant_token: s.participant_token,
    room_name: s.room_name,
  });
}

// app/api/say/route.ts
export async function POST(req) {
  const { session_id, text } = await req.json();
  const r = await fetch(`https://api.2wai.ai/v1/sessions/${session_id}/speak`, {
    method: 'POST',
    headers: {
      Authorization: `Bearer ${process.env.TWAI_SECRET_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ text }),
  });
  return Response.json(await r.json(), { status: r.status });
}

// app/api/end/route.ts (optional — teardown)
export async function POST(req) {
  const { session_id } = await req.json();
  const r = await fetch(`https://api.2wai.ai/v1/sessions/${session_id}`, {
    method: 'DELETE',
    headers: { Authorization: `Bearer ${process.env.TWAI_SECRET_KEY}` },
  });
  return Response.json(await r.json().catch(() => ({})), { status: r.status });
}

Python (FastAPI)

server.py (excerpt)

import os
from fastapi import FastAPI, Request
import httpx

app = FastAPI()
SK = os.environ["TWAI_SECRET_KEY"]
AVATAR = os.environ["AVATAR_ID"]
VOICE = os.environ.get("VOICE_ID")

@app.post("/mint")
async def mint():
    async with httpx.AsyncClient() as c:
        r = await c.post(
            "https://api.2wai.ai/v1/sessions",
            headers={"Authorization": f"Bearer {SK}"},
            json={
                "avatar_id": AVATAR,
                "voice_id": VOICE,
                "mode": "video",
                "transport": "livekit",
            },
        )
        s = r.json()
        return {
            "session_id": s["session_id"],
            "livekit_url": s["livekit_url"],
            "participant_token": s["participant_token"],
            "room_name": s["room_name"],
        }

@app.post("/say")
async def say(req: Request):
    body = await req.json()
    async with httpx.AsyncClient() as c:
        r = await c.post(
            f"https://api.2wai.ai/v1/sessions/{body['session_id']}/speak",
            headers={"Authorization": f"Bearer {SK}"},
            json={"text": body["text"]},
        )
        return r.json()

Three routes: POST /mint (one call to gateway /v1/sessions), POST /say (one call to /v1/sessions/{id}/speak), optional POST /end (one call to DELETE /v1/sessions/{id}). Nothing else lives on the backend for a minimal integration.

Your client: join + subscribe + chat

The client code depends on which track you’re on. Prefer an SDK; the raw path below is the shape every SDK encapsulates.

Web (vanilla JS)

<script src="https://cdn.jsdelivr.net/npm/livekit-client/dist/livekit-client.umd.min.js"></script>
<video id="avatar" autoplay playsinline></video>
<form id="chat"><input id="in"/><button>Send</button></form>
<script type="module">
  const { Room, RoomEvent } = LivekitClient;
  const video = document.getElementById('avatar');

  // 1. Mint.
  const { session_id, livekit_url, participant_token } =
    await (await fetch('/api/mint', { method: 'POST' })).json();

  // 2. Join. adaptiveStream + dynacast MUST be off.
  const room = new Room({ adaptiveStream: false, dynacast: false });
  room.on(RoomEvent.TrackSubscribed, (track) => {
    if (track.kind === 'video') track.attach(video);
    else if (track.kind === 'audio') track.attach();
  });
  room.on(RoomEvent.DataReceived, (payload, _p, _k, topic) => {
    if (topic !== 'lk.chat') return;
    const frame = JSON.parse(new TextDecoder().decode(payload));
    if (frame.text) console.log('reply:', frame.text, 'final:', frame.final);
  });
  await room.connect(livekit_url, participant_token);

  // 3. Chat.
  document.getElementById('chat').addEventListener('submit', async (e) => {
    e.preventDefault();
    const text = document.getElementById('in').value;
    await fetch('/api/say', {
      method: 'POST', headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ session_id, text }),
    });
    document.getElementById('in').value = '';
  });
</script>

Flutter

Use the Flutter SDK — the client code reduces to one widget:

import 'package:twai_avatar_sdk/twai_avatar_sdk.dart';
import 'package:twai_avatar_sdk_livekit/transport0.dart' as t0;

final rest = TwaiApiRestClient(
  baseUrl: Uri.parse('https://my-app.com/api'),  // your mint proxy
  credentials: const AnonymousCredentials(),
);

Future<t0.MintResult> mint() async {
  final r = await rest.createLiveKitSession(
    TwaiApiLiveKitSessionRequest(avatarId: 'avt_...'),
  );
  return t0.MintResultEnabled(
    url: r.livekitUrl.toString(),
    token: r.participantToken,
    room: r.roomName,
  );
}

t0.TwaiTransport0Player(
  mint: mint,
  idleBuilder: (ctx) => NetworkVideoIdle(Uri.parse('https://.../idle.mp4')).build(ctx),
  onReplyText: (e) => print('avatar: ${e.text}'),
);

Full working app: apps/avatar_only_flutter.

iOS (Swift)

import TwaiPlayer

TwaiPlayerView(mint: {
    let s = try await api.createSession(.init(
        avatarId: "avt_...", mode: .video, transport: "livekit"
    ))
    return .enabled(url: s.livekitUrl, token: s.participantToken, room: s.roomName)
})
.onReady { handle in
    try? await handle.send("Hi!")
}

Full sample: ios-native/Samples/HelloAvatar.

Driving a conversation with your own brain

The avatar is just the output end — your backend decides what it says. The reference apps run /mint on page load, join the room, then on each user turn:

1. User types or speaks text (your UI).
2. Your backend /say route forwards to the gateway, OR you call your own LLM first + forward the LLM’s reply as the text parameter.
3. The gateway synthesizes + lip-syncs + streams.

For streaming LLM output (so the avatar starts speaking before your LLM finishes), chunk the reply at sentence boundaries and call /speak for each sentence — see Bring your own brain for the pattern.

Session hygiene

• Create one session per conversation, not one per turn. Reuse via /speak repeatedly.
• End the session when the user leaves (beforeunload → sendBeacon → your /end route).
• Watch x-twai-concurrent-sessions header on /v1/sessions responses so you can back off before hitting your cap.

See Sessions for lifecycle in depth and Limits & concurrency for cap/throttle behavior.

Next

Bring your own brain Swap the gateway's default brain for your own LLM.

Bring your own TTS Push your own synthesized audio instead of using 2wai's TTS.

Voices Register a voice clone from a short reference clip.

Latency Shaving milliseconds off the user-perceived response time.