iOSIntermediate

SwiftUI Quickstart

Build a native iOS voice AI client with SwiftUI.

Recipe prompt

Paste into Cursor, Claude Code, v0, or your coding agent

Use with your coding agent

You are implementing the "SwiftUI Quickstart" recipe in this project.

Read the recipe markdown first:
https://raw.githubusercontent.com/AgoraIO-Conversational-AI/recipe-client-swiftui-quickstart/main/README.md

Use the source repository for cross-reference:
https://github.com/AgoraIO-Conversational-AI/recipe-client-swiftui-quickstart

Build this recipe into the user's app using the markdown as the implementation guide. Inspect related source files through the repository links when the recipe points to them. Ask before installing new dependencies.

Recipe

Rendered from the configured recipe markdown.

Raw

recipe-client-swiftui-quickstart

A native iOS (SwiftUI) voice-agent quickstart for Agora Conversational AI. It talks to a bundled keyless Python backend and renders the live transcript using Agora's official ConversationalAIAPI toolkit — proving the non-web client path on iOS.

┌─────────────────────────┐        ┌──────────────────────────┐
│  ios/ (SwiftUI app)     │  HTTP  │  server/ (FastAPI)       │
│  RTC + RTM 2.0 + toolkit │ ─────► │  token + agent lifecycle │
└─────────────────────────┘        └──────────────────────────┘
        │  audio + RTM transcript ▲
        ▼                         │
        Agora Conversational AI (managed cascade)

Pipeline: DeepgramSTT(nova-3, en) → OpenAI (Agora-managed, keyless) → MiniMaxTTS.
Runs zero-key: the bundled server uses managed vendors, so the only

secrets are your Agora AGORA_APP_ID / AGORA_APP_CERTIFICATE. The app holds no provider key and no App Certificate (the server mints RTC/RTM tokens).

Transcript is rendered via the official toolkit in `.text` mode

(sentence-at-once). A turn has a separate user row and agent row (same turnId, different speaker), so rows are keyed by (turnId, type).

Layout

Path	What
`server/`	keyless FastAPI backend (`GET /get_config`, `POST /startAgent`, `POST /stopAgent`)
`ios/project.yml`	XcodeGen spec (app target, SPM dependencies, Info.plist)
`ios/App/`	`BackendApi`, `AgoraSession`, `CallViewModel`, SwiftUI views
`ios/Tests/`	XCTest for `BackendApi`

Run it

1. Start the server (needs your Agora creds in server/.env.local):

cd server
uv venv venv && . venv/bin/activate
uv pip install -r requirements.txt -r requirements-dev.txt
python src/server.py            # listens on http://localhost:8000

2. Generate + run the app:

cd ios
xcodegen generate
open App.xcodeproj

Pick the App scheme on an iPhone Simulator and run. Tap Connect, allow the microphone, and start talking — the agent replies by voice and the transcript fills in live.

The iOS Simulator shares the host network, so the default AGENT_BACKEND_URL = http://localhost:8000 (set in ios/App/Info.plist) reaches the server directly. App Transport Security already allows http://localhost. To point at a deployed backend, change AGENT_BACKEND_URL in project.yml and regenerate.

Toolchain

Xcode 16+ / iOS 17+ deployment target / Swift 6.
XcodeGen (brew install xcodegen) —

the .xcodeproj is generated and git-ignored; project.yml is the source of truth.

Agora Agent Client Toolkit via SwiftPM, using the

`Package.swift` manifest at exact tag `2.9.0`.

Toolkit 2.9.0 transitively pins RTC AgoraRtcEngine_iOS 4.5.1

(RtcBasic, import AgoraRtcKit) and Signaling AgoraRTM_iOS 2.2.8 (AgoraRTM, import AgoraRtmKit; client type AgoraRtmClientKit).

App code also imports AgoraAgentClientToolkit. Only the former direct RTC

package declaration and local RTM wrapper were removed; the RTC and RTM SDKs remain transitive dependencies of the toolkit.

Agent Client Toolkit dependency

ios/project.yml is the dependency source of truth. XcodeGen resolves the AgoraAgentClientToolkit product at 2.9.0, replacing the previously vendored ConversationalAIAPI source files and local RTM package wrapper.

Tests / CI

iOS: xcodegen generate + xcodebuild build test (XCTest for BackendApi

against a stubbed URLProtocol). The Agora SDKs + toolkit are exercised at build time. CI picks whatever iPhone simulator the runner's Xcode ships.

Server: pytest -q.
Docker: the server/ ships as a backend-only image — docker build . then

the container serves :8000; .github/workflows/docker.yml builds it and smoke-tests /get_config key-less.

See .github/workflows/ci.yml.

Attribution

The app depends on the published AgoraAgentClientToolkit SwiftPM product at tag 2.9.0.