Agora Conversational AI — Interruptions Recipe (Python)

  

The interruptions recipe in the Agora Conversational AI recipes family. Control whether the agent can be interrupted mid-speech: choose from fully interruptible, completely uninterruptable, or keyword-triggered barge-in. The INTERRUPTION_MODE env var sets the Agora interruption config at agent-start time.

This repo ships a zero-key mock LLM endpoint mounted at /llm in the same backend process so you can run the full pipeline immediately, then replace the mock with your own model. STT (Deepgram) and TTS (MiniMax) stay Agora-managed.

Prerequisites

• Python 3.10+
• Bun
• Agora CLI
• ngrok (or any tunnel — the backend must be publicly reachable; Agora cloud calls /llm directly)

Run It

# 1. Install + create the server Python venv
bun run setup

# 2. Add Agora credentials (CLI), or edit server/.env.local by hand
agora login
agora project use <your-project>          # select which project to use (you may have several)
agora project env write server/.env.local # writes App ID/Certificate; keeps your CUSTOM_LLM_* lines

# 3. Expose the backend publicly (Agora cloud calls /llm/chat/completions)
ngrok http 8000

# 4. Add the tunnel URL to server/.env.local (use whatever domain ngrok prints —
#    today that is usually *.ngrok-free.dev)
#    CUSTOM_LLM_URL=https://<your-tunnel>.ngrok-free.dev/llm/chat/completions

# 5. (Optional) Set the interruption mode
#    echo 'INTERRUPTION_MODE=keywords' >> server/.env.local

# 6. Run the backend and web
bun run dev

Open http://localhost:3000 → Start Conversation → speak while the agent is talking to test barge-in.

Working from a clone

After cloning, run bun run setup to create the Python venv, then follow the steps above to add credentials and a tunnel URL.

Services:

Deploy

Deploy web (Next.js) and server (a single publicly reachable FastAPI backend). The mock LLM endpoint is mounted at /llm in the same process, so Agora cloud reaches it at <public-url>/llm/chat/completions. Set AGENT_BACKEND_URL in the web deployment so the Next rewrites reach the backend.

A single-process Docker image is published to ghcr.io/AgoraIO-Conversational-AI/recipe-agent-interruptions on v* tags. It bundles the agent backend and the mock LLM endpoint in one process on port 8000. Point CUSTOM_LLM_URL at <public-url>/llm/chat/completions.

Co-public caveat: the server :8000 is now the public endpoint Agora calls (/llm), so the token endpoints are co-public; the App Certificate is only used in-memory to mint tokens (never on the wire); add auth/rate-limiting before a real deployment.

Environment Variables

Backend env file: `server/.env.example`.

INTERRUPTION_MODE values

The mode mapping lives in server/src/interruption_config.py.

Commands

bun run setup            # install web deps + create server/ venv
bun run dev              # run backend (:8000, serves /llm) + web (:3000)

bun run doctor           # prerequisite check (no creds needed)
bun run doctor:local     # + .env.local + credentials + CUSTOM_LLM_URL checks

bun run verify           # web-only gate (no Agora creds needed)
bun run verify:local     # full local gate: backend compile + smoke tests + web build
bun run clean            # remove venvs and build artifacts

Tests run standalone (no Agora cloud needed): pytest in server/, plus bun run verify in web/. CI runs them on Linux/macOS/Windows × Python 3.10 & 3.13.

Architecture

Browser (localhost:3000)
  │  fetch /api/*
  ▼
Next.js  ──rewrite──▶  Agent backend  (server/, localhost:8000)
                          │  starts agent session (CustomLLM vendor + interruption config)
                          ▼
                       Agora ConvoAI Cloud
                          │  POST <CUSTOM_LLM_URL>   (Authorization: Bearer)
                          ▼
                       Mock LLM endpoint  (mounted at /llm in server/, localhost:8000)
                          ▲  public via ngrok tunnel

The browser only ever calls Next /api/*, which rewrites to the agent backend. The agent backend owns Agora tokens and agent lifecycle. The mock LLM endpoint is mounted at /llm in the same backend; because Agora cloud — not the browser — calls it, the backend must be publicly reachable (ngrok http 8000). See ARCHITECTURE.md.

What You Get

• Next.js web client (React 19 / TypeScript) for RTC/RTM and agent lifecycle
• FastAPI agent backend with token generation and agent session management
• The /api/get_config, /api/startAgent, and /api/stopAgent contract, rewritten by Next to the backend
• INTERRUPTION_MODE → Agora interruption config: interruptible (default), uninterruptable, or keywords (trigger words: "stop", "wait", "hold on")
• The mock LLM endpoint returns a long monologue so you have time to practice interrupting
• Zero-key mock: no external API key needed to run the demo

How It Works

1. Browser loads; Next.js fetches /api/get_config → backend generates a Token007 RTC token and returns channel/UID config.
2. User clicks Start Conversation; Next.js posts /api/startAgent → backend calls Agora ConvoAI to start an agent session using CustomLLM (pointing at CUSTOM_LLM_URL) and applies the interruption config from INTERRUPTION_MODE.
3. Agora ConvoAI Cloud joins the RTC channel, receives audio, and runs STT (Deepgram nova-3).
4. For each LLM turn, Agora POSTs the conversation to CUSTOM_LLM_URL/chat/completions; the mock endpoint streams back a long monologue as an OpenAI SSE response.
5. Agora runs TTS (MiniMax speech26_turbo) and sends audio back to the browser. The interruption config governs whether the user's speech interrupts the agent mid-sentence.
6. User clicks Stop; Next.js posts /api/stopAgent → backend ends the agent session.

Replacing the mock

The mock LLM always returns a long monologue so you can test interruption behavior. To use a real model, replace the body of get_long_reply() in `server/src/llm.py` with your own logic. The endpoint must keep speaking the OpenAI streaming /chat/completions contract. A production endpoint should also validate the Authorization: Bearer header.

Repo Map

Troubleshooting

More Docs

• ARCHITECTURE.md
• AGENTS.md

License

Released under the MIT License.