Flutter iOS quickstart

A complete, verbatim-paste Flutter + iOS integration. Mirrors the toolkit’s apps/hello_avatar/ reference app at the shortest useful shape. Clone, add two git-sourced dependencies, paste a main.dart, run.

Works on Android and web too

The same lib/main.dart runs on Android and web with no code change. Android needs the mic plugin registrant setup (see the SDK’s Flutter page); web needs the mint proxy reachable from the browser.

1. pubspec.yaml

Three load-bearing bits: the SDK packages, iOS dependency pins, and an optional patched speech_to_text for push-to-talk mic.

pubspec.yaml

dependencies:
  flutter:
    sdk: flutter
  twai_avatar_sdk:
    git:
      url: https://github.com/2waiorg/2wai-toolkit-public
      path: packages/twai_avatar_sdk
      ref: main
  twai_avatar_sdk_livekit:
    git:
      url: https://github.com/2waiorg/2wai-toolkit-public
      path: packages/twai_avatar_sdk_livekit
      ref: main

# LOAD-BEARING on iOS — do not bump without testing on a real iPhone.
# livekit_client 2.9+ broke receive-only playout on real iPhones
# (session plays to silence). The pins below are the configuration
# our production integrations ship with.
dependency_overrides:
  livekit_client: 2.8.1
  flutter_webrtc: 1.4.0
  # Only if your app uses push-to-talk mic — a Darwin-patched copy of
  # speech_to_text that doesn't tear down LiveKit playout on stop.
  # speech_to_text:
  #   git:
  #     url: https://github.com/2waiorg/2wai-toolkit-public
  #     path: packages/twai_speech_to_text_patched
  #     ref: main

After flutter pub get:

cd ios && pod install --repo-update

The downgrade needs a fresh Podfile.lock + WebRTC-SDK pod 144.7559.09.

2. lib/main.dart

The whole app. ~200 lines of load-bearing code.

lib/main.dart

import 'dart:async';
import 'package:flutter/material.dart';
import 'package:twai_avatar_sdk/twai_avatar_sdk.dart';
import 'package:twai_avatar_sdk_livekit/transport0.dart' as t0;

// Point this at your running mint proxy. See the mint proxy section
// at the bottom of this page for a 50-line Node reference server you
// can run locally while developing.
const _kMintProxyUrl = String.fromEnvironment(
  'MINT_PROXY_URL',
  defaultValue: 'http://localhost:8787',
);

// Your tenant-scoped avatar_id. This is the one your 2wai account team
// provisions for you — avt_ ids are tenant-scoped, so the one in sample
// snippets won't mint on your sk. See:
//   /concepts/#avatar-ids-are-per-tenant
const _kAvatarId = String.fromEnvironment(
  'AVATAR_ID',
  defaultValue: '<YOUR_AVATAR_ID>',
);

// Idle video that loops before/between sessions. Any public MP4 URL.
// Catalog hydrate (/guides/catalog-hydrate) can resolve this from an
// avatar handle instead of hard-coding.
const _kIdleUrl = String.fromEnvironment(
  'IDLE_URL',
  defaultValue: '<YOUR_IDLE_MP4_URL>',
);

void main() => runApp(const HelloAvatarApp());

class HelloAvatarApp extends StatelessWidget {
  const HelloAvatarApp({super.key});
  @override
  Widget build(BuildContext context) => MaterialApp(
        title: 'Hello Avatar',
        debugShowCheckedModeBanner: false,
        theme: ThemeData(brightness: Brightness.dark, useMaterial3: true),
        home: const _Home(),
      );
}

class _Home extends StatefulWidget {
  const _Home();
  @override
  State<_Home> createState() => _HomeState();
}

class _HomeState extends State<_Home> {
  late final TwaiApiRestClient _rest;
  final _input = TextEditingController();
  t0.LiveKitEngine? _engine;
  t0.EngineState _state = t0.EngineState.idle;

  @override
  void initState() {
    super.initState();
    _rest = TwaiApiRestClient(
      baseUrl: Uri.parse(_kMintProxyUrl),
      credentials: const AnonymousCredentials(),
    );
  }

  // The one closure every Transport 0 integration needs — called by the
  // player on mount / reconnect.
  //
  // voiceId is optional. Omit to use the avatar's default voice (set
  // server-side by its livekit_profile). Pass a vc_... id if you want
  // a specific registered voice.
  Future<t0.MintResult> _mint() async {
    try {
      final m = await _rest.createLiveKitSession(
        TwaiApiLiveKitSessionRequest(avatarId: _kAvatarId),
      );
      return t0.MintResultEnabled(
        url: m.livekitUrl.toString(),
        token: m.participantToken,
        room: m.roomName,
      );
    } on TwaiApiRateLimitedException catch (e) {
      throw t0.MintRateLimited(e.retryAfterSeconds ?? 60);
    } catch (e) {
      throw t0.MintUnavailable('gateway mint failed: $e', cause: e);
    }
  }

  Future<void> _send() async {
    final e = _engine;
    if (e == null || _state != t0.EngineState.ready) return;
    final text = _input.text.trim();
    if (text.isEmpty) return;
    _input.clear();
    try {
      await e.send(text);
    } catch (_) {
      // Fire-and-forget here; show a snackbar / log in production.
    }
  }

  @override
  void dispose() {
    _rest.close();
    _input.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    final ready = _state == t0.EngineState.ready;
    return Scaffold(
      appBar: AppBar(title: const Text('Hello Avatar')),
      // Top-level Listener lifts browser autoplay on web; harmless on native.
      body: Listener(
        behavior: HitTestBehavior.translucent,
        onPointerDown: (_) {
          final e = _engine;
          if (e != null) unawaited(e.resumePlayback());
        },
        child: Column(
          children: [
            Expanded(
              child: AspectRatio(
                aspectRatio: 9 / 16,
                // The whole avatar experience — idle, LiveKit tracks,
                // fades — is this one widget.
                child: t0.TwaiTransport0Player(
                  mint: _mint,
                  idleBuilder: (ctx) =>
                      NetworkVideoIdle(Uri.parse(_kIdleUrl)).build(ctx),
                  onEngine: (e) => setState(() => _engine = e),
                  onState: (s) => setState(() => _state = s),
                ),
              ),
            ),
            SafeArea(
              top: false,
              child: Padding(
                padding: const EdgeInsets.all(12),
                child: Row(
                  children: [
                    Expanded(
                      child: TextField(
                        controller: _input,
                        decoration: InputDecoration(
                          hintText: ready ? 'Say something…' : 'Connecting…',
                          border: const OutlineInputBorder(),
                          isDense: true,
                        ),
                        textInputAction: TextInputAction.send,
                        onSubmitted: (_) => _send(),
                      ),
                    ),
                    const SizedBox(width: 8),
                    IconButton.filled(
                      onPressed: ready ? _send : null,
                      icon: const Icon(Icons.send),
                    ),
                  ],
                ),
              ),
            ),
          ],
        ),
      ),
    );
  }
}

3. Run it

1. Run your mint proxy (see § 5 below).
2. flutter run -d DEVICE_ID — find your iPhone’s id with flutter devices first.
3. Type a message, hit send. Avatar talks within a couple of seconds.

To override the defaults at runtime without editing source:

flutter run -d <device> \
  --dart-define=MINT_PROXY_URL=http://192.168.1.100:8787 \
  --dart-define=AVATAR_ID=avt_... \
  --dart-define=IDLE_URL=https://.../idle.mp4

4. iOS gotchas worth knowing up front

• Mic permission in ios/Runner/Info.plist if adding mic later:

  <key>NSMicrophoneUsageDescription</key>
  <string>Needed to talk to the avatar.</string>

• Push-to-talk gesture: use TwaiMicButton with TwaiMicGesture.tapToToggle on iOS. A raw push-to-talk GestureDetector loses the gesture arena inside scrollable composers.
• Session idle timeout on livealman-style profiles is ~60 seconds of inactivity. Reconnect pattern: remount the player via a ValueKey that bumps when the user sends to a dead session; deliver the queued message from the fresh engine’s onReady callback.
• Don’t force the speaker on reconnect — on livekit_client 2.8.1 + flutter_webrtc 1.4.0, calling Hardware.setSpeakerphoneOn from a reconnect handler makes the native audio config throw and kills playout even though audio still arrives. The remoteOnly native config already routes to the speaker.

5. Mint proxy

The avatar’s avatar_id is per-tenant; the sk_... key that authenticates its mint call is tenant-scoped too and must never ship in your client bundle. The mint proxy is a tiny server that holds your sk_ and forwards /v1/* requests to api.2wai.ai with it attached.

Node reference (~50 lines) Express + fetch; the one we recommend first.

Python reference FastAPI + httpx.

Cloudflare Worker Edge, no server to run.

Set TWAI_SECRET_KEY=sk_test_... in the proxy’s env and point MINT_PROXY_URL at it.

6. If your first mint 404s

Likely no_such_avatar — see tenancy and avatar identity. The short version: avt_... ids are scoped to your account, and the one in a sample snippet won’t mint on your sk. Confirm with your 2wai account team which avt_ ids are registered to your key.

7. Beyond the minimum

Streaming into your app Reconnect, speaking events, data-channel chat envelope.

Catalog hydrate Resolve avatars by human handle instead of hard-coding.

Sessions Lifecycle, timeouts, cleanup.

Reference integration End-to-end walkthrough with the fuller integration shape.