← All templatestemplates / desktop-robot-assistant-with-voice-controlled-manipulation

LiveKit Agentsupdated Jan 20, 2025 · other · other

Desktop Robot Assistant with Voice-Controlled Manipulation

A voice-controlled desktop robot assistant system that performs manipulation tasks like desk cleaning and organization through natural language commands. The system coordinates multiple operators including PID-based motion control, trained ACT policies for pick-and-place operations, and MolmoAct2 VLA for generalized manipulation. The architecture demonstrates distributed robotics with the voice agent, policies, slider control, and VLA running on separate machines connected via LiveKit Portal.

No demo yetView source Fork template

The numbers

latency—

cost / min—

frameworkLiveKit Agents

The stack

telephonyWeb Only

speech-to-textDeepgram Nova-3

llmGPT-4o

text-to-speechCartesia Sonic-3

System prompt

No prompt published.

Config

config.json

{
  "hardware": [
    "SO-101",
    "AprilTag",
    "linear-slider"
  ],
  "policies": [
    "ACT",
    "MolmoAct2"
  ],
  "operators": [
    "move_to",
    "pick_up",
    "put_down"
  ],
  "training_episodes": 200
}

Voice AI recipes, picks, and analysis.

Get the useful new templates plus the occasional teardown of what’s working in production voice AI.