Real-Time Voice Agent with Neural Turn Detection

A production-ready real-time voice agent built with LiveKit Agents framework and AssemblyAI Universal-3 Pro Streaming. Features neural turn detection that distinguishes between mid-sentence pauses and actual end-of-turn signals using acoustic and linguistic cues. Supports mid-session keyterm prompting for domain-specific vocabulary and real-time speaker diarization. Optimized for low-latency conversational experiences with configurable turn detection parameters.

No demo yetView source Fork template

The numbers

latency307ms P50, 1012ms P99

cost / min—

frameworkLiveKit Agents

The stack

telephonyWeb Only

speech-to-textAssemblyAI Universal-2

llmGPT-4o

text-to-speechCartesia Sonic-3

System prompt

No prompt published.

Config

config.json

{
  "features": [
    "mid-session keyterm prompting",
    "real-time speaker diarization",
    "neural turn detection",
    "anti-hallucination"
  ],
  "stt_model": "u3-rt-pro",
  "max_speakers": 2,
  "speaker_labels": false,
  "max_turn_silence": 1200,
  "min_turn_silence": 300,
  "neural_turn_detection": true,
  "keyterms_prompt_support": true,
  "end_of_turn_confidence_threshold": 0.4
}

Voice AI recipes, picks, and analysis.

Get the useful new templates plus the occasional teardown of what’s working in production voice AI.