Real-Time Voice Agent with OpenAI and Twilio

A custom voice AI agent that integrates OpenAI's Realtime API with Twilio Voice services to enable interactive voice conversations. The agent supports real-time audio streaming, automatic speech detection, response cancellation, and natural conversation flow with interrupt handling. Built with FastAPI and WebSocket communication, it can be configured for sales, support, compliance monitoring, and cold calling applications.

No demo yetView source Fork template

The numbers

latency—

cost / min—

frameworkCustom

The stack

telephonyTwilio Voice

speech-to-textOpenAI Whisper

llmGPT-4o

text-to-speechOpenAI TTS-1

System prompt

No prompt published.

Config

config.json

{
  "temperature": 0.2,
  "barge_in": true,
  "audio_format": "G711 ULAW",
  "interrupt_handling": true,
  "session_management": true,
  "system_prompt_file": "prompts/system_prompt.txt"
}

Voice AI recipes, picks, and analysis.

Get the useful new templates plus the occasional teardown of what’s working in production voice AI.