Flutter SDK

The Flutter SDK is the cross-platform path: one codebase, every target 2wai cares about (iOS, Android, Web, Windows, macOS, Linux). Transport 0 (LiveKit) by default, mint-proxy enforced, TwaiTransport0Player widget as the primary consumer surface.

Repo: 2waiorg/2wai-toolkit-public Reference app: apps/avatar_only_flutter — avatar + bring-your-own-brain pattern, ~300 LOC.

Install

Add to pubspec.yaml:

dependencies:
  twai_avatar_sdk:
    git:
      url: https://github.com/2waiorg/2wai-toolkit-public
      path: packages/twai_avatar_sdk
      ref: main
  twai_avatar_sdk_livekit:
    git:
      url: https://github.com/2waiorg/2wai-toolkit-public
      path: packages/twai_avatar_sdk_livekit
      ref: main

# Required pins (iOS audio + LiveKit coexistence)
dependency_overrides:
  livekit_client: 2.8.1
  flutter_webrtc: 1.4.0
  speech_to_text:
    git:
      url: https://github.com/2waiorg/2wai-toolkit-public
      path: packages/twai_speech_to_text_patched
      ref: main

The speech_to_text override is mandatory on iOS — upstream unconditionally deactivates the AVAudioSession on stop, which also kills LiveKit’s playback. Without the override, the avatar goes silent after the first mic turn. See the mic patch doc.

Minimum working integration (~40 LOC)

import 'package:flutter/material.dart';
import 'package:twai_avatar_sdk/twai_avatar_sdk.dart';
import 'package:twai_avatar_sdk_core/twai_avatar_sdk_core.dart';
import 'package:twai_avatar_sdk_livekit/transport0.dart' as t0;

void main() => runApp(const App());

class App extends StatelessWidget {
  const App({super.key});
  @override
  Widget build(BuildContext ctx) => const MaterialApp(home: Scaffold(body: AvatarScreen()));
}

class AvatarScreen extends StatelessWidget {
  const AvatarScreen({super.key});

  @override
  Widget build(BuildContext ctx) {
    final rest = TwaiApiRestClient(
      baseUrl: Uri.parse('https://my-app.com/api'),  // your mint proxy
      credentials: const AnonymousCredentials(),
    );

    Future<t0.MintResult> mint() async {
      final r = await rest.createLiveKitSession(
        TwaiApiLiveKitSessionRequest(avatarId: 'avt_...'),
      );
      return t0.MintResultEnabled(
        url: r.livekitUrl.toString(),
        token: r.participantToken,
        room: r.roomName,
      );
    }

    return t0.TwaiTransport0Player(
      mint: mint,
      idleBuilder: (ctx) => NetworkVideoIdle(
        Uri.parse('https://cdn.example.com/idle.mp4'),
      ).build(ctx),
      onReplyText: (e) => debugPrint('reply: ${e.text}'),
    );
  }
}

That’s it. The widget calls your mint() on mount, joins the LiveKit room, subscribes to the agent’s video + audio tracks, and renders them. The player handles the warm-up window (SEND_GRACE_MS), reconnect, and idle↔speaking crossfade.

Resolve by handle (catalog hydrate)

Instead of hardcoding avatar_id:

final result = await rest.getAvatar('alman', 'dr-alman');
print(result.avatar.displayName);  // "Dr. Alman"
print(result.avatar.avatarId);     // "avt_..."

See Catalog hydrate for the full pattern.

Imperative control: TwaiPlayerHandle

Grab a handle from TwaiTransport0Player.onEngine to send turns programmatically:

t0.Engine? _engine;

t0.TwaiTransport0Player(
  mint: mint,
  onEngine: (e) => setState(() => _engine = e),
  onState: (s) => setState(() => _state = s),
  onSpeaking: (e) => setState(() => _speaking = e.active),
  onReplyText: (e) => print('reply: ${e.text}'),
);

// Later, in a button handler:
await _engine?.send('Hello!');

Mic input

TwaiMicButton(
  transcriber: DefaultTranscriber(),           // SpeechRecognition on web, speech_to_text on native
  enabled: _state == t0.EngineState.ready && !_speaking,
  onTranscript: (text) => _engine?.send(text),
);

Three transcribers ship: DefaultTranscriber() (zero-config), BrowserSpeechTranscriber, SpeechToTextTranscriber, HttpSttTranscriber. See the sample app for wiring.

Reference apps

avatar_only_flutter Avatar + bring-your-own-brain pattern. ~300 LOC.

hello_avatar Smallest working T0 avatar. ~40 LOC of load-bearing code.

twai_api_starter Minimum-viable starter template. Every parameter commented.

twai_api_demo Zero-SDK Flutter harness for the raw /v1/* wire.

Known iOS integration traps

Vendored speech_to_text pin is mandatory. See install block above.
livekit_client pin to 2.8.1 is load-bearing. Newer versions break iOS receive-only audio in ways that only show up on physical devices.
AVAudioSession config happens inside the SDK; don’t configure it yourself in AppDelegate or you’ll fight the SDK.
Full iOS integration gotchas: docs/flutter_consumer_integration_gotchas.md.