Recipes
All recipes
JavaScriptElectronIntermediate

Trak App Voice AI Integration

Control music playback and search tracks using Agora Voice AI.

Source on GitHubView raw recipe

Recipe prompt

Paste into Cursor, Claude Code, v0, or your coding agent
Use with your coding agent
You are implementing the "Trak App Voice AI Integration" recipe in this project.

Read the recipe markdown first:
https://raw.githubusercontent.com/Saarthak1234/Trak-VoiceAI-Demo/main/docs/ai/RECIPE.md

Use the source repository for cross-reference:
https://github.com/Saarthak1234/Trak-VoiceAI-Demo

Build this recipe into the user's app using the markdown as the implementation guide. Inspect related source files through the repository links when the recipe points to them. Ask before installing new dependencies.

Recipe

Rendered from the configured recipe markdown.

Raw
Recipe metadata
FieldValue
recipe_version0.1.0
recipe_statusstable
extension_points
  • api.routes
  • prompts.system
  • pipeline.providers
  • ui.conversation
invariants
  • baseline.official-sample
  • tokens.rtc-rtm
  • lifecycle.strict-mode
  • transcript.uid-remap
stable_contracts
  • env.required
  • api.token
  • api.invite-agent
  • api.stop-conversation

Quickstart Recipe Profile

This repo is a reusable quickstart sample for building browser voice-agent experiences with Agora Conversational AI Engine.

Recipe Role

  • Role: base quickstart recipe.
  • Target audience: developers bootstrapping a production-style Next.js voice agent app.
  • Reuse model: clone, bind project, run, then customize prompt/pipeline/UI.

Recipe Scope

This base recipe provides a copyable browser voice-agent starter with:

  • browser RTC audio and RTM event transport
  • server-side token, invite, stop, and optional custom LLM routes
  • managed default STT, LLM, and TTS provider configuration with Trak music control tags ([PLAY], [TOGGLE], [NEXT], [PREV])
  • pre-call, in-call, transcript, metrics, connection-status UI, and local Electron app control dispatch

Baseline Implementation Guidance

This repository is the official Agora Next.js quickstart baseline for this recipe, integrated with the Trak music player application (trak/). Agents should use this repo's source and progressive disclosure docs as the starting point, then customize.

Do not recreate Agora ConvoAI integration from memory. Provider schemas, SDK builder fields, token behavior, and RTM event details can drift. For a new baseline implementation, follow L1/L2/from_scratch_bootstrap.md while copying verified patterns from this repo.

Extension Points

  • api.routes: add browser-facing routes under app/api, with shared request/response types in types/conversation.ts when the client consumes them.
  • prompts.system: edit ADA_PROMPT and GREETING in app/api/invite-agent/route.ts (includes Trak music playback command tags like [PLAY: <query>], [TOGGLE], [NEXT], [PREV]).
  • pipeline.providers: adjust the DeepgramSTT, OpenAI, and MiniMaxTTS builder chain, or enable the commented BYOK blocks.
  • ui.conversation: customize QuickstartPreCallCard, QuickstartConversationLayout, QuickstartTranscriptPanel, and QuickstartPipelineMetrics.
  • app.integrations: handle parsed transcript tags in ConversationComponent.tsx to control external local applications (e.g. Trak HTTP control endpoint at http://127.0.0.1:3011).

Invariants

  • Keep RtcTokenBuilder.buildTokenWithRtm for RTM-capable tokens.
  • Treat this repo as the official baseline; customize after preserving a working token, invite, RTC, RTM, and transcript flow.
  • Preserve StrictMode isReady guard for join/mic initialization.
  • Preserve UID remap (uid="0") and INTERRUPTED message-list inclusion.
  • Preserve transcript control tag matching ([PLAY], [TOGGLE], [NEXT], [PREV]) for local Trak integration.
  • Keep documentation synchronized when workflows/contracts change.

Stable Contracts

  • GET /api/generate-agora-token returns { token, uid, channel }.
  • POST /api/invite-agent accepts { requester_id, channel_name } and returns the agent id/state payload.
  • POST /api/stop-conversation accepts { agent_id } and treats already-stopping sessions as success.
  • Required env vars are NEXT_PUBLIC_AGORA_APP_ID and NEXT_AGORA_APP_CERTIFICATE.
  • components/LandingPage.tsx owns pre-call bootstrap and RTM client lifecycle.
  • components/ConversationComponent.tsx owns joined-session RTC/toolkit lifecycle and Trak command dispatch.
  • lib/conversation.ts owns transcript normalization helpers.
  • trak/ contains the Electron music player app with HTTP control server (http://127.0.0.1:3011).

Internal / Subject to Change

  • Visual styling and copy in the quickstart UI.
  • The exact reseller defaults for STT, LLM, and TTS models.
  • Connection issue display heuristics and metric chip presentation.

Consumer Onboarding Recipe

  1. Clone or scaffold from template.
  2. Bind Agora project and write .env.local.
  3. Run pnpm run doctor and pnpm run dev.
  4. Validate with pnpm run verify before sharing modifications.
  5. Customize agent behavior and UI using the supported surfaces above.