← All templatestemplates / real-time-multilingual-translation-between-customer-and-contact-center-agent

Customupdated Jun 1, 2025 · other · support

Real Time Multilingual Translation Between Customer and Contact Center Agent

A bidirectional voice translation system that enables real-time conversations between customers and contact center agents speaking different languages. The agent intercepts audio from both parties, translates it using OpenAI Realtime API, and delivers the translated audio to the other party. Supports multiple languages including English, Spanish, French, Mandarin, and Hindi through an IVR menu.

Try the demo View source Fork template

The numbers

latency—

cost / min—

frameworkCustom

The stack

telephonyTwilio Voice

speech-to-textOpenAI gpt-4o-transcribe

llmGPT-4o

text-to-speechOpenAI TTS-1

System prompt

raw_prompt

You are a translation machine. Your sole function is to translate the input text from [CALLER_LANGUAGE] to English.
Do not add, omit, or alter any information.
Do not provide explanations, opinions, or any additional text beyond the direct translation.
You are not aware of any other facts, knowledge, or context beyond translation between [CALLER_LANGUAGE] and English.
Wait until the speaker is done speaking before translating, and translate the entire input text from their turn.
Example interaction:
User: ¿Cuantos días hay en la semana?
Assistant: How many days of the week are there?
User: Tengo dos hermanos y una hermana en mi familia.
Assistant: I have two brothers and one sister in my family.

Config

config.json

{
  "api_port": 5050,
  "agent_language": "English",
  "caller_languages": [
    "English",
    "Spanish",
    "French",
    "Mandarin",
    "Hindi"
  ],
  "uses_task_router": true,
  "uses_twilio_flex": true,
  "uses_twilio_studio": true,
  "dual_openai_connections": true,
  "forward_audio_before_translation": false
}

Voice AI recipes, picks, and analysis.

Get the useful new templates plus the occasional teardown of what’s working in production voice AI.