Multi-User Browser Voice Agent with WebRTC Transport

A Python worker that bridges LiveKit WebRTC rooms with AssemblyAI's Voice Agent API, enabling multi-user voice conversations with an AI agent in browser, mobile, and desktop clients. The worker handles audio transport while AssemblyAI manages the full AI pipeline including speech-to-text, LLM responses, and text-to-speech with neural turn detection and barge-in support.

Try the demo View source Fork template

The numbers

latency—

cost / min—

frameworkCustom

The stack

telephonyWeb Only

speech-to-textAssemblyAI Universal-2

llmGPT-4o

text-to-speechElevenLabs Turbo v2.5

System prompt

raw_prompt

You are a customer support agent for Acme. Speak in 1–2 short sentences. Confirm the user's question before answering.

Config

config.json

{
  "voice_id": "james",
  "barge_in": true,
  "sample_rate": 24000,
  "audio_format": "audio/pcm",
  "num_channels": 1,
  "tool_calling": true,
  "queue_size_ms": 1000,
  "turn_detection": {
    "max_silence": 1500,
    "min_silence": 600,
    "vad_threshold": 0.5,
    "interrupt_response": true
  }
}

Voice AI recipes, picks, and analysis.

Get the useful new templates plus the occasional teardown of what’s working in production voice AI.