Inbound Phone Order Status Agent

A production-ready inbound phone agent that answers calls, transcribes the caller in real time, looks up order status, and offers human handoff. Built with AssemblyAI Universal-3 Pro Streaming for speech recognition, GPT-4o for conversation and function calling, and ElevenLabs for voice synthesis. Maintains 8kHz mulaw audio end-to-end with no resampling, achieving 600-1100ms turn latency.

No demo yetView source Fork template

The numbers

latency600-1100ms turn latency

cost / minA few cents per minute at production scale

frameworkCustom

The stack

telephonyTwilio Voice

speech-to-textAssemblyAI Universal-2

llmGPT-4o

text-to-speechElevenLabs Turbo v2.5

System prompt

No prompt published.

Config

config.json

{
  "tools": [
    {
      "name": "get_order_status",
      "description": "Retrieves the current status of a customer order"
    },
    {
      "name": "transfer_to_human",
      "description": "Transfers the call to a human agent"
    }
  ],
  "server": "FastAPI + WebSockets",
  "stt_model": "u3-rt-pro",
  "stt_encoding": "pcm_mulaw",
  "stt_sample_rate": 8000,
  "tts_output_format": "ulaw_8000"
}

Tools

get_order_statusRetrieves the current status of a customer order
transfer_to_humanTransfers the call to a human agent

Voice AI recipes, picks, and analysis.

Get the useful new templates plus the occasional teardown of what’s working in production voice AI.