LiveKit Voice Agent

A simple CLI-based LiveKit voice agent using Deepgram for STT/TTS and OpenAI for the LLM.

Features

• 🎤 Voice Input via CLI: Speak naturally using voice_agent.py console
• 🔊 Voice Output: Get responses with speech
• 💬 Conversational: Natural language interaction
• ⚙️ STT/TTS with Deepgram: Using Flux for STT and Aura-2 for TTS
• 🤖 LLM with OpenAI: GPT models for understanding
• 🔧 LiveKit Agent: Traditional LiveKit agent architecture

Prerequisites

• Python 3.9 or higher
• UV package manager (or pip)
• API keys for:
  • Deepgram
  • OpenAI
  • LiveKit (Cloud or local server)

Installation

1. Install dependencies:

   uv sync
   # OR
   pip install -r requirements.txt

2. Set up environment variables:

   cp env.template .env

   Edit .env and add your API keys:

   DEEPGRAM_API_KEY=your_deepgram_api_key_here
   OPENAI_API_KEY=your_openai_api_key_here
   LIVEKIT_URL=wss://your-livekit-server.com
   LIVEKIT_API_KEY=your_livekit_api_key_here
   LIVEKIT_API_SECRET=your_livekit_api_secret_here

Usage

Start the Agent

# Start in dev mode (watches for changes)
uv run python voice_agent.py dev

# Or without dev mode
uv run python voice_agent.py

Connect via Console

In another terminal:

# Connect using built-in console command
uv run python voice_agent.py console

This will:

• Connect to the LiveKit room
• Enable your microphone
• Start listening for your voice
• Display transcripts in the terminal
• Play agent responses through your speakers

Configuration

The agent configuration is in voice_agent.py. You can modify:

• STT Model: Deepgram model (default: flux-general-en)
• TTS Model: Deepgram model (default: aura-2-andromeda-en)
• LLM Model: OpenAI model (default: gpt-4o)
• Temperature: LLM randomness (default: 0.7)
• Max Tokens: Response length (default: 1000)
• Key Terms: STT recognition terms

Example:

self.config = {
    'stt_model': 'flux-general-en',
    'stt_keyterms': ['flight', 'departure', 'arrival', 'airline', 'booking', 'travel', 'trip'],
    'stt_eager_eot_threshold': 0.4,
    'tts_model': 'aura-2-andromeda-en',
    'llm_model': 'gpt-4o',
    'llm_temperature': 0.7,
    'llm_max_tokens': 1000
}

Project Structure

.
├── voice_agent.py      # Main voice agent implementation
├── main.py             # Entry point
├── requirements.txt    # Python dependencies
├── pyproject.toml      # UV configuration
├── env.template        # Environment variables template
├── generate_token.py    # Utility for generating LiveKit tokens
└── README.md           # This file

Getting API Keys

Deepgram

1. Go to Deepgram Console
2. Sign up or log in
3. Create a new project
4. Copy your API key

OpenAI

1. Go to OpenAI Platform
2. Sign up or log in
3. Navigate to API Keys
4. Create a new secret key

LiveKit Server

1. Go to LiveKit Cloud
2. Create a new project
3. Copy the WebSocket URL, API Key, and API Secret

Troubleshooting

"Missing environment variables"

• Check your .env file has all required keys
• Verify API keys are correct

"Failed to connect to LiveKit"

• Verify LIVEKIT_URL, API_KEY, and API_SECRET are correct
• Check network connectivity

"Microphone not working"

• Grant microphone permissions
• Ensure microphone is not used by another app
• Try restarting the console command

License

MIT