recipe-agent-langchain

Overview

This recipe shows how to use LangChain as the tool and orchestration layer inside an Agora real-time voice agent.

The key architectural split is:

• Agora handles the real-time voice runtime:
• RTC / RTM
• speech input and output
• turn handling
• session lifecycle
• LangChain handles the agent logic:
• tool selection
• tool execution
• response composition

This pattern is useful when you already have LangChain-style agent logic and want to expose it through a live voice experience instead of a text-only interface.

When to Use This Recipe

Use this recipe when:

• you already have tools, workflows, or retrieval logic in LangChain
• you want to add real-time voice interaction without rebuilding the agent stack
• you want Agora to own the voice runtime while LangChain remains the agent logic layer

Common target use cases include:

• developer assistants
• support assistants
• workflow copilots
• internal knowledge assistants

Architecture

Browser / App
  -> Agora RTC / RTM client
  -> Agora voice agent runtime
  -> Custom LLM callback
  -> LangChain tool layer

Layer responsibilities

Web client

The web client:

• fetches session bootstrap data from /get_config
• joins RTC with the returned RTC token
• logs into RTM with the returned RTM token
• starts the agent with /startAgent
• receives transcripts and agent audio

Agora runtime layer

The Agora runtime:

• receives live user audio
• manages the conversation session
• invokes the configured custom LLM endpoint
• converts returned text back into speech

Custom LLM bridge

The backend exposes an OpenAI-compatible /chat/completions endpoint that acts as a bridge between the Agora runtime and the LangChain agent layer.

LangChain layer

LangChain remains responsible for:

• deciding when tools should be used
• invoking the relevant tool
• composing a short voice-safe response

Repository Shape

This recipe follows a quickstart-style split:

• web/ — real-time voice client
• server/ — session bootstrap, agent lifecycle, and custom LLM bridge

This keeps the integration pattern recognizable for Agora developers while leaving the LangChain layer clearly isolated on the server side.

Validation Path

There are two levels of validation for this recipe:

Repository validation

Run:

cd server
pytest tests -v

cd ../web
bun test
bun run build

This verifies the repository structure, backend contract, and frontend build surface.

Full voice validation

Full voice validation requires:

• valid Agora credentials
• a valid LangChain model provider credential
• a CUSTOM_LLM_BASE_URL that is reachable by the Agora-managed runtime

Important Validation Caveat

A localhost-only backend is not enough for end-to-end voice validation.

Because the Agora runtime calls the custom LLM endpoint from outside your machine, the backend must be reachable at a stable public URL during full validation.

Temporary public tunnels may be useful for short-lived local debugging, but they should not be treated as the stable validation path for this recipe.

For repeatable team validation or production-style testing, deploy the custom LLM endpoint to a persistent public URL and point CUSTOM_LLM_BASE_URL there.

Why This Pattern Matters

This recipe is not about a specific tool example. It is about a reusable integration pattern:

• keep LangChain where it is strongest, in orchestration and tool logic
• let Agora provide the real-time voice runtime layer

That lets developers move from a text agent to a real-time voice agent without replacing their existing LangChain architecture.

Extension Points

This pattern can be adapted to many different LangChain-backed tool layers, including:

• retrieval-backed assistants
• support flows
• internal workflow tools
• multi-step orchestration agents

The important part is the boundary, not the example tool implementation:

• Agora owns voice runtime concerns
• LangChain owns tool-layer concerns