LLM prompt — bring your own brain

Use this prompt when you already have a 2wai avatar screen working (any SDK) and now want to swap the gateway’s default brain for your own LLM — OpenAI, Anthropic, your own fine-tune, a rules engine, anything with a text-in/text-out shape.

The prompt

Copy, paste, fill {PLACEHOLDERS}:

I've got a working 2wai avatar screen in my app. Now I want to drive what the avatar says from MY OWN LLM instead of using the gateway's default brain.

CONTEXT

The pattern:
1. User types or speaks input.
2. My server passes the input to MY LLM (OpenAI / Anthropic / etc.).
3. My LLM streams back a reply.
4. My server calls `POST https://api.2wai.ai/v1/sessions/{session_id}/speak` with the reply text — but **chunked by sentence** as the LLM streams, so the avatar starts speaking before the LLM finishes.

The avatar's audio plays live over the LiveKit room; nothing on the client needs to change.

MY INPUTS

- My LLM provider: {OPENAI / ANTHROPIC / GOOGLE / OPENROUTER / OWN_MODEL}
- My LLM API key env var: {MY_LLM_KEY_ENV_NAME}
- My existing mint-proxy server stack: {NODE / PYTHON / CLOUDFLARE_WORKER}
- My system prompt / persona for the LLM: {ONE_SENTENCE_OR_PASTE_IT_ALL}
- Current session_id storage: {HOW_MY_CLIENT_TRACKS_IT — e.g. "returned from /api/mint and held in a React state hook"}

WHAT I NEED YOU TO DO

1. **Add a `POST /api/chat` route** to my server. It takes `{session_id, text}` from the client, calls my LLM with streaming, and streams `/speak` calls to the gateway at sentence boundaries. Node example (adapt for Python / CF Worker):

   ```ts
   import OpenAI from 'openai';  // or @anthropic-ai/sdk
   const llm = new OpenAI({ apiKey: process.env.{MY_LLM_KEY_ENV_NAME} });

   app.post('/api/chat', async (req, res) => {
     const { session_id, text } = req.body;

     const stream = await llm.chat.completions.create({
       model: 'gpt-4o-mini',   // or your model
       messages: [
         { role: 'system', content: `{MY_SYSTEM_PROMPT}` },
         { role: 'user', content: text },
       ],
       stream: true,
     });

     let buffer = '';
     for await (const chunk of stream) {
       const delta = chunk.choices[0]?.delta?.content ?? '';
       buffer += delta;

       // Flush at sentence boundaries so the avatar starts speaking fast.
       const sentences = buffer.match(/[^.!?]+[.!?]+/g) || [];
       for (const s of sentences) {
         await speakOne(session_id, s.trim());
         buffer = buffer.slice(s.length).trimStart();
       }
     }
     // Flush any trailing fragment that didn't end in punctuation.
     if (buffer.trim()) await speakOne(session_id, buffer.trim());

     res.json({ ok: true });
   });

   async function speakOne(session_id, text) {
     await fetch(`https://api.2wai.ai/v1/sessions/${session_id}/speak`, {
       method: 'POST',
       headers: {
         Authorization: `Bearer ${process.env.TWAI_SECRET_KEY}`,
         'Content-Type': 'application/json',
       },
       body: JSON.stringify({ text }),
     });
   }
   ```

2. **Update my client** to call `/api/chat` with the user's text (instead of `/api/say` directly, if that's what I had). The route takes care of /speak internally.

3. **Interrupt on new input** — if the user starts typing a new message while the avatar is still mid-reply, call `POST https://api.2wai.ai/v1/sessions/{session_id}/interrupt` BEFORE calling my new `/api/chat`. The avatar stops mid-sentence, and the next /speak picks up.

4. **Verify**:
   - Avatar screen loads, agent appears.
   - Type a long question (ask the LLM to write a short poem).
   - Avatar starts speaking within ~2s of hitting Send — NOT waiting for the LLM to finish the whole poem.
   - Sentence breaks sound natural.
   - Interrupting mid-reply with a new message works cleanly.

GOTCHAS TO HONOR

- **Sentence boundaries matter for perceived latency.** Sending the whole LLM output as one /speak call = user waits for the entire LLM generation before the avatar starts talking. Breaking at `.`/`!`/`?` cuts perceived latency to the first-sentence generation time.
- **Don't buffer tiny fragments.** If the LLM streams character-by-character and you send each character to /speak, the TTS pipeline chokes on sub-word fragments. The regex above waits for a sentence terminator. Minimum useful chunk is a sentence (~5-20 words).
- **One session per conversation.** Do not call `/v1/sessions` on each user turn. The same `session_id` carries the whole conversation. Only mint again on reconnect or a new session.
- **TWAI_SECRET_KEY stays server-side.** My client calls `/api/chat` with just text; the server handles the sk_ Authorization header.

DELIVERABLE

- One new server route (`/api/chat`).
- Client update to call it on user turn + call `/api/interrupt` on new-turn-while-speaking.
- One-line README note on which LLM provider + model I'm using.

If you need anything (which LLM I'm actually using, my system prompt, whether I want streaming replies shown as text alongside the avatar, how I track `session_id` on the client), ASK.

Reference docs:
- https://developers.2wai.ai/guides/bring-your-own-brain/
- https://developers.2wai.ai/guides/latency/ — the sentence-boundary tip is called out here too.