Thymia Voice Biomarker Wellness Demo

Real-time voice biomarker analysis during a therapeutic conversation. The AI therapist (Bella) uses live wellness, clinical, and emotion scores to guide the session — celebrating strengths and exploring elevated areas.

What It Does

1. User speaks to Bella (AI wellness therapist) via voice
2. Audio is streamed to Thymia Sentinel API for real-time biomarker analysis
3. Biomarker scores (stress, burnout, fatigue, emotions, etc.) are:

• Injected into the LLM system prompt so Bella can reference them
• Displayed in the client's Thymia tab for the user/operator to see
• Safety analysis shown if concerns are detected

1. Bella actively uses the biomarker data to guide the therapeutic conversation

Architecture

react-video-client-avatar → simple-backend → Agora ConvoAI → server-custom-llm/node
                                                                  ├── go-audio-subscriber (RTC)
                                                                  ├── Thymia module → Thymia Sentinel API
                                                                  └── RTM → Client (biomarkers/progress/safety)

Data flow: simple-backend passes RTC params (app_id, channel, tokens) and the LLM API key through to server-custom-llm/node in each request. The custom LLM uses these to spawn go-audio-subscriber, connect to Thymia, and push biomarkers back via RTM and Agent Update API.

Cleanup flow: When the user ends a call, react-video-client-avatar calls /hangup-agent on simple-backend, which calls Agora's hangup API and then POSTs /unregister-agent to server-custom-llm/node. The custom LLM stops the audio subscriber, disconnects Thymia, and clears all session state.

Shared Projects

This recipe uses the standard sample apps — no special Thymia variants needed. Thymia is enabled via environment variables on the existing projects.

Keys Required

Optional:

Enabling Thymia

Thymia is toggled on in two places:

1. Custom LLM server — set THYMIA_ENABLED=true:

PORT=8100 THYMIA_ENABLED=true node custom_llm.js

The Thymia API key is passed from the backend .env profile through llm_config.params in each request — no env var needed on the custom LLM server.

2. React client — set NEXT_PUBLIC_ENABLE_THYMIA=true in .env.local:

# react-video-client-avatar/.env.local
NEXT_PUBLIC_ENABLE_THYMIA=true

This adds the Thymia tab to the client UI. Without it, the client works normally — video only.

Prerequisites

• Node.js 20+ (custom LLM server) — use nvm use 20
• Node.js 22+ (React client) — use nvm use 22
• Python 3.x (backend)
• Go 1.21+ (only needed if rebuilding the audio subscriber binary)

Setup

1. Build Go Audio Subscriber (one-time)

The Go binary captures RTC audio and pipes it to the Thymia module. You need the binary and native Agora SDK libraries.

cd server-custom-llm/go-audio-subscriber

# Download native Agora SDK libraries (one-time, ~240MB)
cd sdk && bash scripts/install_agora_sdk.sh && cd ..

# Build the Go binary
make

If the binary already exists at bin/audio_subscriber, skip this step.

macOS note: The binary needs agora_sdk_mac/ dylibs at runtime. The audio_subscriber.js wrapper sets DYLD_LIBRARY_PATH automatically.

2. Custom LLM Server

cd server-custom-llm/node
nvm use 20
npm install --legacy-peer-deps

Start with Thymia enabled:

PORT=8100 THYMIA_ENABLED=true node custom_llm.js

No .env file needed — the custom LLM receives the OpenAI API key, Thymia API key, and RTC params from the backend in each request via llm_config.params.

3. Backend .env — Add THYMIA Profile

Add a THYMIA profile to simple-backend/.env. The simple-backend uses profile-prefixed env vars — each key is {PROFILE}_{VAR} (e.g., THYMIA_LLM_URL sets LLM_URL for the THYMIA profile). See AGENT.md for the full list of config keys.

# =====================================================
# THYMIA PROFILE - Voice + Thymia biomarkers via custom LLM
# =====================================================

# Agora credentials
THYMIA_APP_ID=<your-app-id>
THYMIA_APP_CERTIFICATE=
THYMIA_AGENT_AUTH_HEADER=<your-auth-header>

THYMIA_ENABLE_MLLM=false

# Thymia API key — passed to custom LLM via llm_config.params
THYMIA_THYMIA_API_KEY=<your-thymia-api-key>

# LLM — point to custom LLM server
# LLM_URL: your custom LLM's /chat/completions endpoint
# LLM_VENDOR: "custom" tells Agora ConvoAI to add turn_id + timestamp
# LLM_STYLE: "openai" for OpenAI-compatible request/response format
THYMIA_LLM_URL=<your-custom-llm-url>/chat/completions
THYMIA_LLM_VENDOR=custom
THYMIA_LLM_STYLE=openai
THYMIA_LLM_MODEL=gpt-5.1
THYMIA_LLM_API_KEY=<your-openai-key>

# TTS + ASR
THYMIA_TTS_VENDOR=rime
THYMIA_TTS_KEY=<your-rime-key>
THYMIA_TTS_VOICE_ID=astra
THYMIA_ASR_VENDOR=ares
THYMIA_ENABLE_AIVAD=true

# Agent behavior
THYMIA_MAX_HISTORY=5
THYMIA_IDLE_TIMEOUT=120

# Greeting + Prompt (see Prompt section below — paste as single line with \n)
THYMIA_DEFAULT_GREETING=Hi there! I'm Bella. I'd love to have a quick chat and learn a bit about how you're doing. What's your name?
THYMIA_DEFAULT_PROMPT=<see Prompt section below — paste as single line with \n>

`THYMIA_LLM_URL` must be reachable from the Agora ConvoAI Engine (cloud). For local development, use a Cloudflare tunnel (see Local Development below). For production, use your deployed server URL.

4. Python Backend

cd agent-samples/simple-backend
python3 local_server.py

5. React Client

cd agent-samples/react-video-client-avatar
nvm use 22
npm run dev

The Thymia tab is enabled by NEXT_PUBLIC_ENABLE_THYMIA=true in .env.local.

6. Connect

Open the React client, enter THYMIA in the Server Profile field, and click Start Conversation.

Or use the URL shortcut: `?profile=THYMIA`

Local Development

When running the custom LLM server locally, the Agora ConvoAI Engine (cloud) can't reach localhost. Use a Cloudflare tunnel to expose it:

# Install (macOS)
brew install cloudflare/cloudflare/cloudflared

# Start tunnel pointing to custom LLM server
cloudflared tunnel --url http://localhost:8100
# Outputs a URL like https://xxx-yyy.trycloudflare.com

Use that tunnel URL as THYMIA_LLM_URL in your backend .env:

THYMIA_LLM_URL=https://xxx-yyy.trycloudflare.com/chat/completions

Not needed in production if the custom LLM server is deployed with a public URL.

Greeting

Hi there! I'm Bella. I'd love to have a quick chat and learn a bit about how you're doing. What's your name?

Prompt

Set as DEFAULT_PROMPT for the THYMIA profile (THYMIA_DEFAULT_PROMPT) in the backend .env (paste as single line with \n for newlines):

You are Bella, a warm wellness therapist. Your goal is to have a therapeutic conversation that helps improve the user's wellbeing, guided by real-time voice biomarker data.

WORD LIMITS: MAX 30 WORDS per response. Keep it conversational.

CONVERSATION FLOW:
1. Start by learning their name, then use it naturally throughout
2. Ask how their overall mood has been lately
3. Ask about their energy levels and how they've been sleeping
4. Gently explore what's been on their mind or causing stress
5. Ask how they cope when things get tough or feelings are difficult
- Move through these topics naturally — don't rush or follow a rigid script
- Be curious and empathetic
- Keep responses short (MAX 30 WORDS)
- Ask one question at a time, let them talk

BIOMARKER DATA — THIS IS YOUR SUPERPOWER, USE IT:
- A voice analysis system runs during this call and biomarker results update continuously in a system message
- You MUST actively reference the biomarker data in your responses when it is available
- Every 2-3 responses, bring up something from the biomarkers — this is the whole point of the conversation
- You cannot hear the user's voice — the data comes from a separate analysis system

INTERPRETING WELLNESS & CLINICAL SCORES:
- Under 10% = very positive — celebrate these!
- 10-20% = normal, healthy range
- 20-30% = slightly elevated, worth noting
- Over 30% = elevated, explore and work on improving
- Balance positive reinforcement with therapeutic exploration

INTERPRETING EMOTION SCORES:
- Emotions show real-time affect — they change moment to moment
- High neutral is normal baseline, not a concern
- Note shifts in emotions during conversation

HOW TO REFERENCE BIOMARKERS:
- DO NOT say "I can hear" or "your voice sounds" — you cannot hear them
- You CAN share exact percentages if the user asks — be transparent with the data
- DO connect the biomarkers to the conversation
- DO celebrate the positives

SAFETY GUIDANCE:
- You may receive a [Safety Guidance] message with specific suggestions — follow these

IMPORTANT:
- Be warm, conversational, not clinical
- Never pretend you can hear the user's voice
- When no biomarker data is available yet, just have a normal conversation
- When biomarker data IS available, you MUST use it to guide the therapy

What to Expect

1. 0-10s: Bella greets user, audio subscriber connects to channel
2. 10-30s: Progress indicators appear in Thymia tab (speech seconds counting up)
3. 30-60s: First biomarker scores arrive — emotions, wellness (Helios), clinical (Apollo)
4. 60s+: Bella starts referencing biomarker data in conversation
5. Safety analysis: If concerns detected, safety section appears at top of Thymia tab

Thymia Tab Sections

• Progress — Speech seconds / trigger threshold per biomarker set
• Safety Analysis — Alert level, concerns, guidance, urgency (color-coded)
• Emotions — Real-time affect: angry, disgusted, fearful, happy, neutral, sad, surprised
• Helios (Wellness) — Distress, stress, burnout, fatigue, low self-esteem
• Apollo (Clinical) — Depression probability, anxiety probability, severity indicators

Troubleshooting

• No biomarkers appearing: Check custom LLM logs for Thymia connection errors. Verify THYMIA_THYMIA_API_KEY is set in the backend .env profile and THYMIA_ENABLED=true on the custom LLM server.
• Thymia tab not showing: Ensure NEXT_PUBLIC_ENABLE_THYMIA=true in client .env.local and restart dev server.
• Bella not referencing data: Check custom LLM logs for AgentUpdate entries — biomarkers should be pushed via Agent Update API.
• Audio subscriber not connecting: Verify Go binary exists at go-audio-subscriber/bin/audio_subscriber. On macOS, verify sdk/agora_sdk_mac/ has the dylib files.
• "Cannot find module" errors: Run npm install --legacy-peer-deps in server-custom-llm/node/. Make sure Node.js 20+ is active (nvm use 20).
• Agent not starting: Check backend logs for the /start-agent request. Verify LLM_URL (i.e. THYMIA_LLM_URL) is reachable from the Agora cloud.
• RTM messages not reaching client: The custom LLM initializes RTM from request params on first call. Check custom LLM logs for "RTM init" messages.
• ASR with null key: If using Deepgram, set THYMIA_ASR_KEY. Or use THYMIA_ASR_VENDOR=ares (Agora built-in, no key needed).