M5 StickS3 Agora Voice Agent

# M5 StickS3 Agora Voice Agent

Build a pocket ESP32-S3 voice agent on M5Stack StickS3 (and optional StopWatch) that talks to Agora Conversational AI through a small FastAPI control service.

## What you get

- ESP-IDF firmware using Agora RTSA Lite for realtime audio (G.722 uplink).
- On-device LVGL UI with session states and volume control.
- A LAN FastAPI service that issues RTC/RTM tokens and starts/stops the Conversational AI Agent.
- Default **international (Global / US)** Agora path with managed Deepgram STT, OpenAI LLM, and MiniMax TTS.
- Optional China (Shengwang) path for workshop deployments.

## Hardware

| Board | Notes |
| --- | --- |
| M5StickS3 | Primary target: 135x240 display, ES8311 + AW8737, buttons A/B |
| M5StopWatch | Same esp32s3 firmware; runtime board profile adjusts round UI |

You need Wi-Fi reachability from the device to the machine running the FastAPI service.

## Architecture

```text
StickS3 firmware  --HTTP LAN-->  FastAPI control service  --REST-->  Agora Conversational AI
      |                                 |
   Agora RTSA                     App ID + App Certificate
   (G.722 audio)                  (secrets stay server-side)
```

Device endpoints used by firmware:

- `GET /get_config` ? channel, uid, short-lived token
- `POST /startAgent` ? start Conversational AI Agent
- `POST /stopAgent` ? stop agent

Half-duplex today: mic pauses while the agent speaks. No barge-in / AEC in this sample.

## Prerequisites

- Agora **international** console project with Conversational AI (ConvoAI) enabled and App Certificate on
- ESP-IDF **v6.0.3**
- Python 3.10+ for the FastAPI service
- M5StickS3 (or StopWatch) over USB

## Quick start (international / default)

1. Clone this repository and install ESP-IDF v6.0.3.
2. Configure the control service:

```bash
cd service/agent-quickstart-python/server
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# set AGORA_APP_ID, AGORA_APP_CERTIFICATE
# leave AGORA_AREA=us (default)
python src/server.py
```

3. Copy `main/voice_config.local.example.h` to `main/voice_config_local.h` and set Wi-Fi SSID/password plus `VOICE_SERVER_URL` to your LAN host (for example `http://192.168.0.101:8001`). Keep the default global area code (`0xFFFFFFFF`) for international projects.
4. Build and flash:

```bash
idf.py set-target esp32s3
idf.py build flash monitor
```

5. On the device: **A** starts/stops a session, **B** cycles speaker volume.

## China region alternate

If you use a Shengwang.cn project instead:

- Set `AGORA_AREA=cn` and `MINIMAX_API_KEY` in the server `.env`.
- In `voice_config_local.h`, set `#define VOICE_AREA_CODE 0x00000001U`.
- Log in with Agora CLI `--region cn` and use the China console.

Device and server regions must match.

## Controls and limits

- Idle listen timeout: 15 s; reply wait timeout: 30 s.
- App Certificate and vendor keys never leave the FastAPI process.
- Swap boards carefully: audio pins, display layout, and input mapping are board-profile specific.

## License

Project code is MIT (`LICENSE`). Third-party components (Agora RTSA binaries, M5/LVGL snapshots, libg722, fonts, Agora quickstart subtree) are listed in `NOTICE.md`; keep those terms when redistributing.