# M5 StickS3 Agora Voice Agent

> Pocket ESP32-S3 voice agent on M5StickS3 with Agora Conversational AI.

Build and flash an ESP-IDF voice client on M5Stack StickS3 (and StopWatch) that talks to Agora Conversational AI through a small FastAPI control service over your LAN.

- Author: Dylan Luo
- Updated: 2026-11-04
- Difficulty: Intermediate
- Categories: voice-ai, rtc
- Platforms: ESP-IDF, Python, ESP32
- Use cases: Voice AI, Embedded, Hardware
- Capabilities: Conversational AI, Voice Agent, RTC, RTSA
- [Recipe page](https://recipes.agora.io/recipes/m5-sticks3-voice-agent)
- [Implementation repository](https://github.com/Ocean-lhy/RTE2026-StickS3-Demo)
- [Source recipe](https://github.com/Ocean-lhy/RTE2026-StickS3-Demo/blob/main/docs/ai/RECIPE.md)
- [Raw source Markdown](https://raw.githubusercontent.com/Ocean-lhy/RTE2026-StickS3-Demo/main/docs/ai/RECIPE.md)
- [All recipes](https://recipes.agora.io/llms.txt)

## Implementation guide

# M5 StickS3 Agora Voice Agent

Build a pocket ESP32-S3 voice agent on M5Stack StickS3 (and optional StopWatch) that talks to Agora Conversational AI through a small FastAPI control service.

## What you get

- ESP-IDF firmware using Agora RTSA Lite for realtime audio (G.722 uplink).
- On-device LVGL UI with session states and volume control.
- A LAN FastAPI service that issues RTC/RTM tokens and starts/stops the Conversational AI Agent.
- Default **international (Global / US)** Agora path with managed Deepgram STT, OpenAI LLM, and MiniMax TTS.
- Optional China (Shengwang) path for workshop deployments.

## Hardware

| Board | Notes |
| --- | --- |
| M5StickS3 | Primary target: 135x240 display, ES8311 + AW8737, buttons A/B |
| M5StopWatch | Same esp32s3 firmware; runtime board profile adjusts round UI |

You need Wi-Fi reachability from the device to the machine running the FastAPI service.

## Architecture

```text
StickS3 firmware  --HTTP LAN-->  FastAPI control service  --REST-->  Agora Conversational AI
      |                                 |
   Agora RTSA                     App ID + App Certificate
   (G.722 audio)                  (secrets stay server-side)
```

Device endpoints used by firmware:

- `GET /get_config` ? channel, uid, short-lived token
- `POST /startAgent` ? start Conversational AI Agent
- `POST /stopAgent` ? stop agent

Half-duplex today: mic pauses while the agent speaks. No barge-in / AEC in this sample.

## Prerequisites

- Agora **international** console project with Conversational AI (ConvoAI) enabled and App Certificate on
- ESP-IDF **v6.0.3**
- Python 3.10+ for the FastAPI service
- M5StickS3 (or StopWatch) over USB

## Quick start (international / default)

1. Clone this repository and install ESP-IDF v6.0.3.
2. Configure the control service:

```bash
cd service/agent-quickstart-python/server
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# set AGORA_APP_ID, AGORA_APP_CERTIFICATE
# leave AGORA_AREA=us (default)
python src/server.py
```

3. Copy `main/voice_config.local.example.h` to `main/voice_config_local.h` and set Wi-Fi SSID/password plus `VOICE_SERVER_URL` to your LAN host (for example `http://192.168.0.101:8001`). Keep the default global area code (`0xFFFFFFFF`) for international projects.
4. Build and flash:

```bash
idf.py set-target esp32s3
idf.py build flash monitor
```

5. On the device: **A** starts/stops a session, **B** cycles speaker volume.

## China region alternate

If you use a Shengwang.cn project instead:

- Set `AGORA_AREA=cn` and `MINIMAX_API_KEY` in the server `.env`.
- In `voice_config_local.h`, set `#define VOICE_AREA_CODE 0x00000001U`.
- Log in with Agora CLI `--region cn` and use the China console.

Device and server regions must match.

## Controls and limits

- Idle listen timeout: 15 s; reply wait timeout: 30 s.
- App Certificate and vendor keys never leave the FastAPI process.
- Swap boards carefully: audio pins, display layout, and input mapping are board-profile specific.

## License

Project code is MIT (`LICENSE`). Third-party components (Agora RTSA binaries, M5/LVGL snapshots, libg722, fonts, Agora quickstart subtree) are listed in `NOTICE.md`; keep those terms when redistributing.


## Recipe prompt

You are implementing the "M5 StickS3 Agora Voice Agent" recipe in this project.

Read the recipe markdown first:
https://raw.githubusercontent.com/Ocean-lhy/RTE2026-StickS3-Demo/main/docs/ai/RECIPE.md

Use the source repository for cross-reference:
https://github.com/Ocean-lhy/RTE2026-StickS3-Demo

Build this recipe into the user's app using the markdown as the implementation guide. Inspect related source files through the repository links when the recipe points to them. Ask before installing new dependencies.