# Cross-Session Memory

> Remember returning users across separate calls.

Capture conversation history before the agent stops and re-inject it next time so it remembers a returning user.

- Author: Agora
- Updated: 2026-06-12
- Difficulty: Intermediate
- Categories: voice-ai
- Platforms: Python, TypeScript
- Use cases: Personalization, Memory, Voice AI
- Capabilities: Conversational AI, Voice Agent, Memory, SQLite
- [Recipe page](https://recipes.agora.io/recipes/cross-session-memory)
- [Implementation repository](https://github.com/AgoraIO-Conversational-AI/recipe-agent-memory)
- [Source recipe](https://github.com/AgoraIO-Conversational-AI/recipe-agent-memory/blob/main/README.md)
- [Raw source Markdown](https://raw.githubusercontent.com/AgoraIO-Conversational-AI/recipe-agent-memory/main/README.md)
- [All recipes](https://recipes.agora.io/llms.txt)

## Use with Agora CLI

agora init my-cross-session-memory --recipe cross-session-memory

## Implementation guide

# Agora Conversational AI — Memory Recipe (Python)

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE)
[![Python](https://img.shields.io/badge/python-%3E%3D3.10-blue)](https://www.python.org/)
[![Bun](https://img.shields.io/badge/bun-latest-black)](https://bun.sh/)

The **cross-session memory** recipe in the Agora Conversational AI recipes family.
A returning user (identified by a name handle entered before the call) is remembered
across sessions — the agent re-injects their conversation history via `system_messages`
so it can greet them naturally and recall what was said before. Fully **zero-key** —
OpenAI is Agora-managed (no `OPENAI_API_KEY` required unless you bring your own account).

**Pipeline:** `DeepgramSTT(nova-3, en)` → `OpenAI` (warm assistant + memory context) → `MiniMaxTTS`

## Prerequisites

- [Python 3.10+](https://www.python.org/)
- [Bun](https://bun.sh/)
- [Agora CLI](https://github.com/AgoraIO/cli) — makes generating an App ID + App Certificate easy

The same commands work on macOS, Linux, and Windows. On macOS/Linux, setup uses
`python3`; on Windows, it uses the Python launcher (`py`) or `python`. WSL and
virtualenv activation are not required.

## Run It

```bash
# 1. Install web deps + create the Python venv
bun run setup

# 2. Add Agora credentials (CLI), or edit server/.env.local by hand
agora login
agora project use <your-project>          # select which project to use
agora project env write server/.env.local # writes App ID + Certificate

# 3. Run backend + web
bun run dev
```

Open [http://localhost:3000](http://localhost:3000), enter your name (optional),
then click **Start Conversation**. On a second call with the same name the agent
will greet you using what you told it before.

### Working from a clone

If you cloned this repo (rather than scaffolding via the Agora CLI), the steps
above are complete as written: `bun run setup` creates the Python venv and
installs web dependencies, then `bun run dev` brings up both services. You
still need Agora credentials in `server/.env.local` before a conversation can connect.

Services:

- Frontend — http://localhost:3000
- Backend — http://localhost:8000
- Mock LLM — N/A (managed OpenAI, no local service)
- API docs — http://localhost:8000/docs

## Deploy

Deploy `web` (Next.js) and `server` (a reachable FastAPI backend). Set
`AGENT_BACKEND_URL` in the web deployment so the Next rewrites reach the backend.

The SQLite memory file (`memory.db`) is local to wherever the server process runs.
For multi-instance deployments, point `MEMORY_DB_PATH` at a shared volume.

A backend-only Docker image is published to
`ghcr.io/AgoraIO-Conversational-AI/recipe-agent-memory` on `v*` tags.
It exposes **BACKEND-ONLY** (:8000). No separate LLM container is needed —
OpenAI is Agora-managed.

## Environment variables

Backend env file: [`server/.env.example`](https://github.com/AgoraIO-Conversational-AI/recipe-agent-memory/blob/main/server/.env.example).

| Variable | Required | Default | Notes |
| --- | :---: | :---: | --- |
| `AGORA_APP_ID` | ✅ | — | Agora Console → Project → App ID |
| `AGORA_APP_CERTIFICATE` | ✅ | — | Agora Console → Project → App Certificate |
| `OPENAI_MODEL` | | `gpt-4o-mini` | OpenAI model |
| `OPENAI_API_KEY` | | — | Optional — Agora manages the OpenAI key by default (keyless). Set only if your account requires it. |
| `MEMORY_DB_PATH` | | `memory.db` | Path to the SQLite file (relative to `server/`, or absolute). In Docker: `/tmp/memory.db`. |
| `MEMORY_MAX_TURNS` | | `20` | Rolling cap on stored turns per user handle. |
| `AGENT_GREETING` | | built-in | Optional opening line override |

## Commands

```bash
bun run setup            # install web deps + create server/ venv
bun run dev              # run backend (:8000) + web (:3000)

bun run doctor           # prerequisite check (no creds needed)
bun run doctor:local     # + .env.local + credentials checks

bun run verify           # web-only gate (no Agora creds needed)
bun run verify:local     # full local gate: backend compile + smoke tests + web build
bun run clean            # remove venvs and build artifacts
```

Tests run standalone (no Agora cloud needed): `pytest` in `server/`, plus
`bun run verify` in `web/`. CI runs them on Linux/macOS/Windows × Python 3.10 & 3.13.

## Architecture

```
Browser (localhost:3000)
  │  fetch /api/*
  ▼
Next.js  ──rewrite──▶  Agent backend  (server/, localhost:8000)
                          │  loads SQLite memory for user handle
                          │  starts agent session with memory-injected system_messages
                          ▼
                       Agora ConvoAI Cloud
                          │  Deepgram STT (managed, nova-3, en)
                          │  OpenAI (Agora-managed, keyless)
                          │  MiniMax TTS (managed)
                          ▼
                       User hears warm, context-aware responses

On stop:  session.get_history() ──▶  SQLite (memory.db)
          (captured BEFORE session.stop())
```

No separate `llm/` service — OpenAI is Agora-managed and requires no API key.
See [ARCHITECTURE.md](https://github.com/AgoraIO-Conversational-AI/recipe-agent-memory/blob/main/ARCHITECTURE.md).

## What You Get

- **Cross-session memory**: the user enters a name handle before the call; prior conversation turns are loaded from SQLite and re-injected via `llm.system_messages` so the agent greets them by name and recalls earlier topics.
- A **Next.js** web client (:3000) that drives the RTC/RTM lifecycle and only ever calls `/api/*`.
- A **FastAPI** agent backend (:8000) that owns Agora token generation, agent session lifecycle, and the SQLite memory store.
- The `/api/get_config` · `/api/startAgent` · `/api/stopAgent` contract between the web client and the backend (Next rewrites, no Route Handlers).
- **Managed keyless OpenAI** — Agora-managed, no `OPENAI_API_KEY` required.
- **Zero-key** setup — the full pipeline runs with no LLM API key by default.

## How It Works

1. The browser calls `/api/get_config`; the backend mints an Agora token from
   `AGORA_APP_ID` + `AGORA_APP_CERTIFICATE`.
2. The user enters an optional name handle. The browser calls `/api/startAgent`
   with `userKey` in the body.
3. The backend calls `get_history(handle)` on the SQLite store, loads any prior
   turns for that handle, and builds `system_messages = [base_system_prompt, memory_context]`.
4. The agent session starts with those system messages; the model greets the
   user using recalled context.
5. The user speaks; Agora runs STT → OpenAI → MiniMax TTS → audio back to the user.
6. `/api/stopAgent` is called (end-call button). The server calls
   `await session.get_history()` **before** `session.stop()` — the only window
   where history is available — then persists the turns to SQLite for that handle.

**Limitation:** if the session times out due to idle silence (30 s), the SDK
stop path bypasses the history-capture block. End the call via the button to
ensure memory is saved.

## Repo Map

- `web/` — Next.js frontend (:3000); RTC/RTM lifecycle and UI.
- `server/` — FastAPI agent backend (:8000); Agora tokens, agent lifecycle, SQLite memory.
- `ARCHITECTURE.md` — system shape and component boundaries.
- `AGENTS.md` — guide for coding agents working in this repo.

## Troubleshooting

| Problem | Fix |
| --- | --- |
| Agent does not recall previous conversation | Make sure you used the same name handle in both sessions and ended the first call via the end-call button (not by closing the tab). |
| `memory.db` grows large | Reduce `MEMORY_MAX_TURNS` or delete the file to reset all memory. |
| Local calls fail under a global proxy (Clash, etc.) | Configure your proxy to send `127.0.0.1`, `localhost`, and RFC-1918 ranges DIRECT. |

## More Docs

- [ARCHITECTURE.md](https://github.com/AgoraIO-Conversational-AI/recipe-agent-memory/blob/main/ARCHITECTURE.md)
- [AGENTS.md](https://github.com/AgoraIO-Conversational-AI/recipe-agent-memory/blob/main/AGENTS.md)

## License

Released under the [MIT License](https://github.com/AgoraIO-Conversational-AI/recipe-agent-memory/blob/main/LICENSE).


## Recipe prompt

You are implementing the "Cross-Session Memory" recipe in this project.

Read the recipe markdown first:
https://raw.githubusercontent.com/AgoraIO-Conversational-AI/recipe-agent-memory/main/README.md

Use the source repository for cross-reference:
https://github.com/AgoraIO-Conversational-AI/recipe-agent-memory

Build this recipe into the user's app using the markdown as the implementation guide. Inspect related source files through the repository links when the recipe points to them. Ask before installing new dependencies.