# Delegated Reasoning

> Keep a voice agent responsive while a second model reasons and calls tools.

A full-duplex GPT Live voice agent that delegates reasoning and tool use to a separate Responses model (gpt-6-astra) over MCP, so the conversation stays live while a slow tool runs.

- Author: Agora
- Updated: 2026-09-25
- Difficulty: Advanced
- Categories: voice-ai
- Platforms: TypeScript
- Use cases: Voice AI, Conversational AI, Tool Use
- Capabilities: Conversational AI, Voice Agent, OpenAI GPT Live, Delegated Reasoning, MCP, Next.js
- [Recipe page](https://recipes.agora.io/recipes/agent-delegation)
- [Implementation repository](https://github.com/AgoraIO-Conversational-AI/recipe-agent-delegation)
- [Source recipe](https://github.com/AgoraIO-Conversational-AI/recipe-agent-delegation/blob/main/docs/ai/RECIPE.md)
- [Raw source Markdown](https://raw.githubusercontent.com/AgoraIO-Conversational-AI/recipe-agent-delegation/main/docs/ai/RECIPE.md)
- [All recipes](https://recipes.agora.io/llms.txt)

## Use with Agora CLI

agora init my-agent-delegation --recipe agent-delegation

## Implementation guide

# Delegated reasoning: a voice agent that stays responsive while a second model thinks

A full-duplex **GPT Live** voice agent that hands the hard thinking and tool use to a
separate **Responses model** (`gpt-6-astra`) over the Agora Conversational AI Engine. The
delegate drives a slow tool over **MCP**, so the voice loop keeps talking and listening while
the reasoning happens off the critical path.

Built on the official [`agora-agents`](https://www.npmjs.com/package/agora-agents) SDK and
Next.js, adapted from the
[OpenAI GPT Live + Next.js sample](https://github.com/AgoraIO-Community/OpenAI-Agora-Voice-Agents-NextJS).

## The idea

A single real-time model that also reasons and calls tools goes silent on the turn it stops to
think. Splitting the work fixes that:

- **GPT Live (front model)** owns the live audio: it listens, talks, backchannels, and decides
  when a turn needs real work. Set with `.withMllm(new OpenAIGPTLive(...))`.
- **The Responses model (`gpt-6-astra`)** is a reasoning delegate the front model consults
  mid-conversation. It picks candidate cities, calls the tools, compares results, and composes
  the answer, which comes back out through GPT Live's voice. Enabled with
  `delegation: 'responses'` + `responsesModel`.
- **Tools over MCP**: a small trip-planner MCP server exposes `get_forecast` (real Open-Meteo
  lookups, a few seconds) and `save_trip` (writes the chosen trip to disk). Attached with
  `mcpServers` + `.withTools(true)`.

## The scenario

> "I want to go somewhere sunny this weekend."

The front model backchannels and says it's checking; the delegate looks up several cities in
one `get_forecast` call, compares them, and recommends one; you pick a city and it calls
`save_trip`; change your mind and it reuses the same trip id so there's one record, not two.

## Prerequisites

- An Agora project with the **GPT Live (live-models) preview** feature enabled — App ID + App
  Certificate.
- An **OpenAI key** with access to GPT Live and the delegate model (`gpt-6-astra`).
- Node 20+ and `pnpm`. Python 3.10+ to run the MCP server.

## Run it

### 1. The MCP server (tools)

```bash
cd mcp-server
pip install -r requirements.txt
python server.py            # binds 0.0.0.0:8787, MCP endpoint at /mcp
```

The agent runs in **Agora's cloud**, so it must reach this server over the public internet.
Expose it at a public HTTPS URL (a tunnel such as cloudflared/ngrok, or a real deployment) and
use that URL as `NEXT_MCP_ENDPOINT` below. `save_trip` writes to `mcp-server/output/` by
default (`OUTPUT_DIR` to change).

### 2. The app

```bash
cp env.local.example .env.local   # fill in App ID, certificate, OpenAI key, NEXT_MCP_ENDPOINT
pnpm install
pnpm run dev                       # http://localhost:3000
```

Open the app, allow the microphone, and talk. Leave `NEXT_MCP_ENDPOINT` empty to demo
delegation without tools.

## How the delegation is wired

In `app/api/invite-agent/route.ts`:

```ts
const agent = new Agent({
  client,
  advancedFeatures: { enable_rtm: true, enable_tools: Boolean(MCP_ENDPOINT) },
  parameters: { audio_scenario: 'chorus', data_channel: 'rtm', enable_error_message: true, enable_metrics: true },
}).withMllm(
  new OpenAIGPTLive({
    apiKey: OPENAI_API_KEY,
    model: 'gpt-live-1',
    prompt: FRONT_INSTRUCTIONS,
    greeting: GREETING,
    delegation: 'responses',           // delegate reasoning + tools
    responsesModel: 'gpt-6-astra',      // the delegate model
    outputBufferMs: -1,                 // lowest first-audio latency
    ...(MCP_ENDPOINT && {
      mcpServers: [{ name: 'trip-planner', endpoint: MCP_ENDPOINT, transport: 'streamable_http', timeout_ms: 30000 }],
    }),
    params: { responses_params: { instructions: BACKEND_INSTRUCTIONS } },
  }),
);
const readyAgent = MCP_ENDPOINT ? agent.withTools(true) : agent;   // expose MCP tools to the delegate
```

The front and backend models are prompted separately: the front prompt is the conversational
persona and when to hand off; the backend prompt (sent via `responses_params.instructions`) is
the trip-planning logic.

## What "working" means

- **It does not guess while waiting.** No temperature, forecast, or recommendation is spoken
  before the tool returns; the front model fills the wait ("let me hand this to gpt-6").
- **It stays interactive during the wait.** Speak again while the lookup runs and it responds,
  because the reasoning is off the voice loop.
- **A correction leaves one record.** After "actually, Seville" `save_trip` reuses the trip id,
  so there is one file, not two.
- **It does not re-delegate to repeat itself.** "What did you say again?" produces no new tool
  calls.

## Notes and limitations

- **GPT Live is a preview provider**, routed to Agora's preview gateway by the SDK. Your App ID
  must have the `live-models` preview feature.
- **No avatar with GPT Live.** The `agora-agents` SDK does not allow an avatar with an MLLM
  (`withMllm()` + `withAvatar()` throws); avatars require the cascading ASR + LLM + TTS
  pipeline. This recipe is voice-only.
- `save_trip` is a stand-in for any real action — booking, ordering, filing a ticket. Swap that
  one tool and the rest is unchanged.


## Recipe prompt

You are implementing the "Delegated Reasoning" recipe in this project.

Read the recipe markdown first:
https://raw.githubusercontent.com/AgoraIO-Conversational-AI/recipe-agent-delegation/main/docs/ai/RECIPE.md

Use the source repository for cross-reference:
https://github.com/AgoraIO-Conversational-AI/recipe-agent-delegation

Build this recipe into the user's app using the markdown as the implementation guide. Inspect related source files through the repository links when the recipe points to them. Ask before installing new dependencies.