Agora Conversational AI — RPG Gaming Recipe (Python)

  

The RPG gaming recipe in the Agora Conversational AI recipes family. A voice RPG where a managed-LLM Dungeon Master narrates the adventure and calls game tools mounted in the same backend process. The player speaks; the DM resolves every mechanic — dice rolls, combat, loot, inventory — through 6 self-contained MCP tools backed by SQLite. STT (Deepgram) and TTS (MiniMax) are Agora-managed.

This recipe is zero-key: OpenAI is Agora-managed (no OPENAI_API_KEY needed, though you may supply your own). The FastMCP game server is mounted in-process — one backend, one port (:8000). The full pipeline runs locally with only Agora credentials and a public tunnel.

Distinct from `recipe-agent-tool-calling`: in that recipe tools run inside the llm/ endpoint. Here Agora cloud orchestrates them on a FastMCP server mounted inside the API server — Agora cloud calls it directly at MCP_ENDPOINT (<public-url>/mcp).

Prerequisites

• Python 3.10+
• Bun
• Agora CLI — generates App ID + Certificate
• ngrok — the /mcp path must be publicly reachable so

Agora cloud can call it

Run It

# 1. Install Python venv + web deps
bun run setup

# 2. Add Agora credentials to server/.env.local
agora login
agora project use <your-project>
agora project env write server/.env.local

# 3. Expose the backend publicly — /mcp is served by the same process
ngrok http 8000

# 4. Set MCP_ENDPOINT in server/.env.local (use whatever domain ngrok prints)
#    MCP_ENDPOINT=https://<your-tunnel>.ngrok-free.dev/mcp

# 5. Run backend + frontend
bun run dev

Open http://localhost:3000 → Start Conversation → say "I want to be a warrior" to create your hero, then explore and fight.

Working from a clone

If you cloned this repo (rather than scaffolding via the Agora CLI), the steps above are complete as written: bun run setup creates the Python venv and installs web dependencies, then bun run dev brings up the backend and frontend. You still need Agora credentials in server/.env.local and a public MCP_ENDPOINT tunnel before a conversation can connect.

Services:

• Frontend — http://localhost:3000
• Backend + MCP game server — http://localhost:8000
• API docs — http://localhost:8000/docs
• MCP endpoint — http://localhost:8000/mcp

Deploy

Deploy web (Next.js) and server (a single publicly reachable FastAPI backend). Set AGENT_BACKEND_URL in the web deployment so the Next rewrites reach the backend.

The backend must be publicly reachable so Agora cloud can call /mcp. A single Docker image is published to ghcr.io/AgoraIO-Conversational-AI/recipe-agent-rpg on v* tags. It runs one process on port 8000 with the FastMCP game server mounted at /mcp.

Environment variables

Backend env file: `server/.env.example`.

Commands

bun run setup            # install web deps + create server/ venv
bun run dev              # run backend (:8000) + web (:3000)

bun run doctor           # prerequisite check (no creds needed)
bun run doctor:local     # + .env.local + credentials + MCP_ENDPOINT checks

bun run verify           # web-only gate (no Agora creds needed)
bun run verify:local     # full local gate: backend compile + web build
bun run clean            # remove venvs and build artifacts

Tests run standalone: pytest in server/, plus bun run verify in web/. CI runs them on Linux/macOS/Windows × Python 3.10 & 3.13.

Architecture

Browser (localhost:3000)
  │  fetch /api/*
  ▼
Next.js  ──rewrite──▶  Agent backend  (server/, localhost:8000)
                          │  starts agent session (Dungeon Master LLM + mcp_servers)
                          │  also serves /mcp  (FastMCP game server, in-process)
                          ▼
                       Agora ConvoAI Cloud
                          │  user speech → Deepgram STT (managed)
                          │  Dungeon Master LLM (managed OpenAI, keyless) → emits tool call
                          │  POST <MCP_ENDPOINT>   (streamable-http)
                          ▼
                       FastMCP game server  (mounted at /mcp, same process)
                          │  public via ngrok tunnel on :8000
                          │  resolves dice/combat/inventory → returns result
                          ▼
                       Agora ConvoAI Cloud → DM narrates outcome
                                          → MiniMax TTS (managed) → user hears speech
                                          → RTM transcript / metrics → web UI

The browser only ever calls Next /api/*, which rewrites to the agent backend. The agent backend owns Agora tokens, agent lifecycle, and the FastMCP game server — all in one process on port 8000. See ARCHITECTURE.md.

What You Get

• A voice RPG where a managed-LLM Dungeon Master narrates the adventure and

calls game tools — no UI to click, no state to manage client-side.

• A managed-LLM DM that narrates and calls 6 self-contained MCP tools: dice

rolling, character creation, combat rounds, spells, fleeing, and inventory reads.

• SQLite backs dice, combat, and inventory — no external game server or

database required.

• Zero-key: OpenAI is Agora-managed and the game engine needs no external

credentials. The full pipeline runs locally with only Agora credentials and a public tunnel.

Each tool opens its own SQLite connection, resolves the full action (including dice rolls and counterattacks), and returns a plain-English result for the DM to narrate. No chaining — one player utterance maps to at most one tool call.

How It Works

1. The browser calls /api/get_config; the backend mints an Agora token.
2. The browser joins the RTC channel, then calls /api/startAgent; the backend

starts an agent session using the managed OpenAI vendor with mcp_servers pointing at the public MCP_ENDPOINT (<tunnel>/mcp) and enable_tools: true.

1. The user speaks (e.g. "I want to be a warrior"). Agora runs STT (Deepgram)

and sends the transcript to the managed Dungeon Master LLM.

1. The DM decides to call create_character("warrior"). Agora cloud issues a

streamable-HTTP request to MCP_ENDPOINT. The FastMCP server (mounted at /mcp in the same process) runs the tool and returns a narrative result string.

1. Agora feeds the tool result back to the DM LLM, which narrates it (e.g.

"You are a warrior with 30 HP…"). Agora runs TTS (MiniMax) and plays it back.

1. Later tools (start_encounter, attack, cast_spell, flee) resolve combat

in the same way — each tool is self-contained (dice rolled inside game.py, no tool-call chaining).

1. /api/stopAgent ends the session.

Repo Map

• web/ — Next.js frontend (:3000); RTC/RTM lifecycle and UI.
• server/ — FastAPI agent backend (:8000); Agora tokens, Dungeon Master agent

lifecycle, and the FastMCP game server (mounted at /mcp).

• server/src/game.py — pure game engine (SQLite, no MCP dependency, fully unit-testable).
• server/src/mcp_server.py — FastMCP wrapper exposing 6 game tools.
• ARCHITECTURE.md — system shape and component boundaries.
• AGENTS.md — guide for coding agents working in this repo.

Troubleshooting

More Docs

• ARCHITECTURE.md
• AGENTS.md

License

Released under the MIT License.