Voice to Video LiveKit Agent (Node.js)

Overview

In this article, you’ll create a realtime TruGen avatar session using LiveKit (Node.js).

The avatar is powered by a LiveKit agent worker and configured using environment variables.

What you’ll build

A realtime avatar setup where:

• LiveKit handles the voice session
• TruGen renders the avatar
• OpenAI powers the voice AI
• Users can speak and receive responses in real time

Prerequisites

• TruGen API key
• TruGen Avatar ID
• LiveKit API key, secret, and URL
• OpenAI API key
• Node.js (v18+)

Project setup

1. Clone the example

git clone https://github.com/trugenai/trugen-examples.git
cd trugen-examples/trugen-livekit-plugin

2. Install dependencies

npm install @livekit/agents @livekit/agents-plugin-openai @livekit/agents-plugin-trugen dotenv

3. Configure environment variables

Create a .env file:

# TruGen Config
TRUGEN_API_KEY="..."
TRUGEN_AVATAR_ID="..."

# OpenAI Config
OPENAI_API_KEY="..."

# LiveKit Config
LIVEKIT_API_KEY="..."
LIVEKIT_API_SECRET="..."
LIVEKIT_URL="..."

These values configure the avatar, AI model, and LiveKit connection.

4. Create the agent

Create a file agent.js:

import 'dotenv/config';

import { defineAgent, cli, ServerOptions, voice } from '@livekit/agents';
import * as openai from '@livekit/agents-plugin-openai';
import * as trugen from '@livekit/agents-plugin-trugen';
import { fileURLToPath } from 'url';

console.log("Avatar ID:", process.env.TRUGEN_AVATAR_ID);

export default defineAgent({
  entry: async (ctx) => {

    console.log(" Agent starting...");

    // 1. Create agent 
    const agent = new voice.Agent({
      instructions: `
        You are a friendly AI assistant.
        Keep responses short and clear.
      `,
    });

    // 2. Create session (OpenAI realtime)
    const session = new voice.AgentSession({
      llm: new openai.realtime.RealtimeModel({
        voice: 'alloy',
      }),
    });

    // 3. Connect to LiveKit
    await ctx.connect();

    // 4. Start agent
    await session.start({
      agent,
      room: ctx.room,
    });

    console.log(" Agent connected to room");

    // 5. Try to start Trugen avatar 
    try {
      const avatar = new trugen.AvatarSession({
        avatarId: process.env.TRUGEN_AVATAR_ID,
      });

      await avatar.start(session, ctx.room);
      console.log(" Avatar started");

    } catch (e) {
      console.log(" Avatar failed (ignored):", e.message);
    }

    // 6. Greeting
    session.generateReply({
      instructions: 'Say: Hello! I am your AI assistant. How can I help you today?',
    });

  },
});

// Run app
cli.runApp(new ServerOptions({
  agent: fileURLToPath(import.meta.url),
}));

5. Start the agent worker

node agent.js dev

This starts the LiveKit agent worker.

6. Test the avatar

Open the LiveKit playground:

https://agents-playground.livekit.io/

Then:

• Sign in with your LiveKit credentials
• Click Connect
• Speak to the agent

Result

• A realtime avatar session is created
• Voice input is processed by the agent
• The avatar responds in real time (if avatarId is valid)

Next steps

• Customize agent behavior
• Add conversation memory
• Replace OpenAI with Groq or Gemini
• Build a frontend UI