Recipes
योगदान दें
शुरू करें
Recipes पर लौटें

Retrieval-Augmented Generation

एजेंट के जवाब अपने ज्ञान भंडार पर आधारित करें।

Python / TypeScriptमध्यमAgora
GitHub

बनाने की गाइड

इस गाइड में

Agora Conversational AI — RAG Recipe (Python)

License: MIT Python Bun

The rag recipe in the Agora Conversational AI recipes family. The agent's LLM stage is pointed at the /llm endpoint that lives inside the same backend process: on each user query the endpoint retrieves the best-matching document from a small in-code corpus and grounds its reply in it ("Based on our docs: …"). STT (Deepgram nova-3) and TTS (MiniMax) stay Agora-managed.

This repo ships a zero-key mock RAG endpoint so you can run the full STT → RAG LLM → TTS pipeline immediately. Retrieval is real code; generation is mocked. Swap CORPUS and retrieve() in server/src/llm.py for a real vector store when you are ready.

Prerequisites

  • Python 3.10+
  • Bun
  • Agora CLI — makes generating an App ID + App Certificate easy
  • ngrok — the backend (including /llm) must be publicly reachable so Agora cloud can call it

The same commands work on macOS, Linux, and Windows. On macOS/Linux, setup uses python3; on Windows, it uses the Python launcher (py) or python. WSL and virtualenv activation are not required.

Run It

bash
# 1. Install Python venv + web deps
bun run setup

# 2. Add Agora credentials (CLI), or edit server/.env.local by hand
agora login
agora project use <your-project>          # select which project to use (you may have several)
agora project env write server/.env.local # writes App ID/Certificate; keeps your CUSTOM_LLM_* lines

# 3. Expose the backend publicly (Agora cloud calls /llm/chat/completions directly)
ngrok http 8000

# 4. Add the tunnel URL to server/.env.local (use whatever domain ngrok prints —
#    today that is usually *.ngrok-free.dev)
#    CUSTOM_LLM_URL=https://<your-tunnel>.ngrok-free.dev/llm/chat/completions

# 5. Run the backend and web
bun run dev

Open http://localhost:3000 → Start Conversation → ask about refunds, business hours, shipping, or warranty.

Working from a clone

If you cloned this repo (rather than scaffolding via the Agora CLI), the steps above are complete as written: bun run setup creates the Python venv and installs web dependencies, then bun run dev brings up the backend and web. You still need Agora credentials in server/.env.local and a public CUSTOM_LLM_URL tunnel before a conversation can connect.

Services:

Deploy

Deploy web (Next.js) and server (a single publicly reachable FastAPI process that also serves /llm/chat/completions). Set AGENT_BACKEND_URL in the web deployment so the Next rewrites reach the backend.

A Docker image is published to ghcr.io/AgoraIO-Conversational-AI/recipe-agent-rag on v* tags. It runs a single process on port 8000 — no second port needed. Point CUSTOM_LLM_URL at <public-url>/llm/chat/completions. A local docker run still needs a tunnel, because Agora cloud cannot reach localhost.

co-public caveat: the backend serves both agent tokens and the /llm endpoint on the same public URL. This is intentional for the mock: in production, move RAG logic to a dedicated service and point CUSTOM_LLM_URL there.

Environment variables

Backend env file: server/.env.example.

VariableRequiredDefaultNotes
AGORA_APP_ID✅—Agora Console → Project → App ID
AGORA_APP_CERTIFICATE✅—Agora Console → Project → App Certificate
CUSTOM_LLM_URL✅—Public URL of the /llm/chat/completions endpoint. Agora cloud calls it; cannot be localhost.
CUSTOM_LLM_API_KEY✅any-key-hereForwarded by Agora cloud as Authorization: Bearer. Required by the CustomLLM vendor.
CUSTOM_LLM_MODELrag-mockModel name passed to your endpoint
AGENT_GREETINGbuilt-inOptional opening line override
RAG_TOP_K1Number of corpus docs to retrieve per query
AGENT_BACKEND_URL (web deploy)✅—Required in a deployed web app when proxying to the backend

Commands

bash
bun run setup            # install web deps + create server/ venv
bun run dev              # run backend (:8000, including /llm) + web (:3000)

bun run doctor           # prerequisite check (no creds needed)
bun run doctor:local     # + .env.local + credentials + CUSTOM_LLM_URL checks

bun run verify           # web-only gate (no Agora creds needed)
bun run verify:local     # full local gate: backend compile + smoke tests + web build
bun run clean            # remove venv and build artifacts

Tests run standalone (no Agora cloud needed): pytest in server/, plus bun run verify in web/. CI runs them on Linux/macOS/Windows × Python 3.10 & 3.13.

Architecture

text
Browser (localhost:3000)
  │  fetch /api/*
  ▼
Next.js  ──rewrite──▶  Agent backend  (server/, localhost:8000)
                          │  starts agent session (CustomLLM vendor)
                          │  also serves /llm/chat/completions (same process)
                          ▼
                       Agora ConvoAI Cloud
                          │  POST <CUSTOM_LLM_URL>   (Authorization: Bearer)
                          ▼
                       /llm endpoint  (server/src/llm.py, mounted at /llm)
                          ▲  public via ngrok tunnel (ngrok http 8000)

The browser only ever calls Next /api/*, which rewrites to the agent backend. The agent backend owns Agora tokens and agent lifecycle. The RAG LLM endpoint is mounted at /llm in the same process — expose port 8000 publicly and set CUSTOM_LLM_URL=<tunnel>/llm/chat/completions. See ARCHITECTURE.md.

What You Get

  • A Next.js web client (:3000) that drives the RTC/RTM lifecycle and only ever calls /api/*.
  • A FastAPI agent backend (:8000) that owns Agora token generation and the agent session lifecycle.
  • The /api/get_config · /api/startAgent · /api/stopAgent contract between the web client and the backend (Next rewrites, no Route Handlers).
  • The /llm endpoint retrieves the best-matching document from an in-code corpus and grounds the reply in it; Agora cloud receives only the final spoken response.
  • A zero-key mock so the full pipeline runs with no LLM API key.

How It Works

  1. The browser calls /api/get_config, which Next rewrites to the backend; the backend mints an Agora token from AGORA_APP_ID + AGORA_APP_CERTIFICATE.
  2. The browser joins the RTC channel, then calls /api/startAgent; the backend starts an agent session using the CustomLLM vendor pointed at CUSTOM_LLM_URL.
  3. The user speaks. Agora runs STT (Deepgram nova-3), then sends the transcript to the /llm endpoint as an OpenAI POST /chat/completions request, forwarding CUSTOM_LLM_API_KEY as Authorization: Bearer.
  4. Inside the endpoint, retrieve() scores the user query against the in-code CORPUS and returns the top-RAG_TOP_K documents. run_agent_turn() grounds the reply in those documents and streams the result in the OpenAI SSE chunk format ("Based on our docs: …").
  5. Agora runs TTS (MiniMax) on the streamed reply and plays it back in the channel.
  6. /api/stopAgent ends the session.

Replacing the mock

Swap CORPUS and retrieve() in server/src/llm.py for a real vector store (e.g. ChromaDB, pgvector, Pinecone). The run_agent_turn() function and the OpenAI streaming contract must remain unchanged. A production endpoint should also validate the Authorization: Bearer header.

Repo Map

  • web/ — Next.js frontend (:3000); RTC/RTM lifecycle and UI.
  • server/ — FastAPI agent backend (:8000); Agora tokens + agent lifecycle, CustomLLM vendor, and /llm mock endpoint.
  • ARCHITECTURE.md — system shape and component boundaries.
  • AGENTS.md — guide for coding agents working in this repo.

Troubleshooting

ProblemFix
Agent starts but never speaksCUSTOM_LLM_URL is not public or omits /llm/chat/completions. Use your ngrok URL.
doctor:local warns about localhostReplace the local URL with your public tunnel URL.
Local calls fail / hang under a global proxy (Clash, etc.)Configure it to send 127.0.0.1, localhost, and RFC-1918 ranges DIRECT.

More Docs

License

Released under the MIT License.