Skip to main content

Configuration Reference

Complete reference for all ConversationConfig options in the UG Labs JavaScript SDK.

Required Options​

apiUrl​

apiUrl: string

WebSocket URL for the UG Labs API server.

// Staging environment
apiUrl: 'wss://pug.stg.uglabs.app'

// Production environment
apiUrl: 'wss://pug.uglabs.app'

apiKey​

apiKey: string

Your API key for authentication. Obtain from UG Labs Console.

federatedId​

federatedId: string

Unique identifier for the user/player. Used for:

  • Conversation continuity across sessions
  • User-specific memory and context
  • Analytics and tracking
federatedId: 'user-123'
federatedId: `${tenantId}:${userId}`  // Multi-tenant pattern

hooks​

hooks: {
  onStateChange?: (state: ConversationState) => void
  onTextMessage?: (event: TextEvent) => void
  onDataMessage?: (event: DataEvent) => void
  onError?: (error: ConversationError) => void
  // ... more hooks
}

Event callbacks for receiving conversation updates. At minimum, you should implement onTextMessage and onError.

Content Configuration​

prompt​

prompt?: string

The system prompt that defines the AI assistant's behavior and personality.

prompt: `You are a friendly math tutor for kids.
Always explain concepts simply.
Use encouraging language.`

Note: Ignored if configRef is provided.

configRef​

configRef?: string

Reference to a deployed configuration from the UG Labs Console. When set, prompt, utilities, voiceProfile, and safetyPolicy are loaded from the backend.

// Created via console.stg.uglabs.app/playground → Deploy
configRef: 'char_abc123_v2'

context​

context?: Record<string, string | number | boolean>

Variables that can be used in prompt templates using {{variable}} syntax.

prompt: 'You are helping {{userName}} with {{topic}}.',
context: {
  userName: 'Alice',
  topic: 'math homework',
}

Capabilities Configuration​

capabilities​

capabilities?: {
  audio?: boolean      // Enable TTS playback (default: true)
  subtitles?: boolean  // Enable real-time captions (default: true)
  viseme?: boolean     // Enable lip-sync data (default: false)
  avatar?: boolean     // Enable avatar animations (default: true)
}

Controls which playback features are enabled. Disable unused features for better performance.

// Text-only mode
capabilities: { audio: false, subtitles: false }

// Voice with captions
capabilities: { audio: true, subtitles: true }

// Full avatar experience
capabilities: { audio: true, subtitles: true, avatar: true, viseme: true }

inputCapabilities​

inputCapabilities?: {
  audio?: boolean  // Enable microphone input (default: true)
  text?: boolean   // Enable text input (default: true)
}

Controls which input modalities are enabled.

// Text-only input (no microphone)
inputCapabilities: { audio: false, text: true }

// Voice-only input
inputCapabilities: { audio: true, text: false }

Voice Configuration​

voiceProfile​

voiceProfile?: VoiceProfile

Configuration for text-to-speech voice synthesis.

ElevenLabs Provider​

voiceProfile: {
  provider: 'elevenlabs',
  voice_id: 'your-voice-id',
  speed: 1.0,           // 0.7 - 1.2
  stability: 0.5,       // 0.0 - 1.0
  similarity_boost: 0.5 // 0.0 - 1.0
}
ParameterRangeDescription
speed0.7 - 1.2Speaking speed
stability0.0 - 1.0Voice consistency (higher = more stable)
similarity_boost0.0 - 1.0Voice similarity to original

Deepdub Provider​

voiceProfile: {
  provider: 'deepdub',
  voice_id: 'your-voice-id',
  deepdub_model: 'dd-etts-2.5',  // or 'dd-etts-1.1'
  deepdub_tempo: 1.0,            // 0.0 - 2.0
  deepdub_variance: 0.5,         // 0.0 - 1.0
  deepdub_locale: 'en-US',
}
ParameterRangeDescription
deepdub_modeldd-etts-1.1, dd-etts-2.5TTS model version
deepdub_tempo0.0 - 2.0Speaking speed
deepdub_variance0.0 - 1.0Expressiveness
deepdub_localeBCP-47 codeOutput language/locale

Utilities Configuration​

Utilities add AI analysis to conversations. They must be both defined AND activated.

utilities​

utilities?: Record<string, Classify | Extract | Reference | null>

Define utility configurations:

utilities: {
  // Classifier utility
  sentiment: {
    type: 'classify',
    classification_question: 'What is the sentiment of: {{user_input}}?',
    answers: ['positive', 'negative', 'neutral'],
    additional_context: 'Focus on emotional tone.',  // Optional
  },

  // Extractor utility
  topic: {
    type: 'extract',
    extract_prompt: 'Extract the main topic in 2-5 words.',
  },

  // Reference to backend-defined utility
  myUtility: { reference: 'util_abc123' },

  // Disable a utility
  disabledUtility: null,
}

onOutputUtilities​

onOutputUtilities?: string[]

Recommended. Utilities to run after the assistant generates a response.

onOutputUtilities: ['sentiment', 'topic']

Results arrive via onDataMessage hook:

onDataMessage: (event) => {
  console.log(event.data.sentiment)  // 'positive'
  console.log(event.data.topic)      // 'weather forecast'
}

onInputUtilities​

onInputUtilities?: string[]

Utilities to run before the prompt is rendered. Results are available in prompt context.

Warning: Adds latency to responses. Use sparingly.

onInputUtilities: ['intent_classifier']

// In prompt:
prompt: 'User intent: {{intent_classifier}}. Respond accordingly.'

onInputNonBlockingUtilities​

onInputNonBlockingUtilities?: string[]

Utilities to run in parallel with LLM processing. Results are not available in prompt context.

onInputNonBlockingUtilities: ['analytics_tracker']

Audio Recording Configuration​

recordingConfig​

recordingConfig?: AudioRecordingConfig

Configuration for microphone audio capture.

recordingConfig: {
  sampleRate: 48000,       // Hz (default: 41000)
  channels: 1,             // Mono (default: 1)
  echoCancellation: true,  // Browser echo cancellation
  noiseSuppression: true,  // Browser noise suppression
  autoGainControl: true,   // Browser auto gain
  timeslice: 100,          // Chunk interval in ms
}

languageCode​

languageCode?: string

BCP-47 language code hint for speech-to-text transcription.

languageCode: 'en'    // English
languageCode: 'he'    // Hebrew
languageCode: 'es'    // Spanish

Safety & Policy​

safetyPolicy​

safetyPolicy?: string

Identifier for content safety policy to apply.

safetyPolicy: 'kids-safe'

Note: Ignored if configRef is provided.

Image Generation​

The SDK supports AI image generation via REST API. No WebSocket connection required - you can generate images without calling initialize().

Providers​

Two providers are available:

  • Bria - Fast generation with optional tailored mode
  • Replicate - Flexible with LoRA support

generateImage()​

Generic method that works with both providers:

const result = await manager.generateImage({
  provider: 'bria',  // or 'replicate'
  prompt: 'A sunset over mountains',
  negative_prompt: 'blurry, low quality',
  aspect_ratio: '16:9',
});

console.log(result.image);  // Base64 encoded image

generateImageWithBria()​

Bria-specific generation:

const result = await manager.generateImageWithBria({
  prompt: 'A futuristic city',
  generation_type: 'fast',  // 'fast' or 'tailored'
  aspect_ratio: '1:1',
  seed: 12345,              // For reproducible results
  inference_steps: 30,
  guidance_scale: 7.5,
});

generateImageWithReplicate()​

Replicate-specific generation with LoRA support:

const result = await manager.generateImageWithReplicate({
  prompt: 'Portrait in watercolor style',
  model: 'flux-schnell',
  aspect_ratio: '3:4',
});

Image-to-Image Generation​

Modify an existing image by providing a base64-encoded source image:

const result = await manager.generateImage({
  provider: 'bria',
  prompt: 'Make it look like winter',
  image: base64EncodedImage,  // Your source image
  strength: 0.7,              // 0.0 = no change, 1.0 = complete transformation
});
ParameterTypeDescription
imagestringBase64 encoded source image
strengthnumberTransformation strength (0.0 - 1.0)

Image Generation Parameters​

ParameterBriaReplicateDescription
prompt✓✓Text description of desired image
negative_prompt✓✓What to avoid in the image
aspect_ratio✓✓Image dimensions (e.g., '16:9', '1:1')
seed✓✓For reproducible results
inference_steps✓✓Quality vs speed tradeoff
guidance_scale✓✓Prompt adherence (higher = stricter)
model✓✓Specific model to use
generation_type✓-'fast' or 'tailored' (Bria only)
lora_weights-✓URL to LoRA weights (Replicate only)
lora_scale-✓LoRA strength, typically -1 to 3

Receiving Images in Conversation​

When images are generated during a conversation, receive them via the onImageChange hook:

hooks: {
  onImageChange: (event) => {
    console.log('New image:', event.image);  // Base64 image data
  },
}

Event Hooks Reference​

onStateChange​

onStateChange?: (state: ConversationState) => void

Called when conversation state changes.

type ConversationState =
  | 'uninitialized'     // Before initialize()
  | 'initializing'      // Connecting
  | 'idle'              // Ready for interaction
  | 'listening'         // VAD active, waiting for speech
  | 'userSpeaking'      // User speech detected
  | 'waiting'           // Processing on server
  | 'playing'           // Playing audio response
  | 'paused'            // Playback paused
  | 'processing_complete'
  | 'interrupted'       // User interrupted AI
  | 'error'

onTextMessage​

onTextMessage?: (event: TextEvent) => void

Called for each streaming text chunk from the assistant.

interface TextEvent {
  event: 'text'
  text: string  // Text chunk (not full response)
  kind: 'interact'
  uid: string
}

Important: Text arrives in chunks. Concatenate for full response.

onDataMessage​

onDataMessage?: (event: DataEvent) => void

Called when utility results are available.

interface DataEvent {
  event: 'data'
  data: Record<string, any>  // { utilityName: result }
  kind: 'interact'
  uid: string
}

onError​

onError?: (error: ConversationError) => void

Called when an error occurs.

interface ConversationError {
  type: 'mic_denied' | 'network_timeout' | 'network_error' | 'server_error' | 'decode_error'
  message: string
  originalError?: Error
}

onNetworkStatusChange​

onNetworkStatusChange?: (isReady: boolean) => void

Called when WebSocket connection status changes.

onSubtitleChange​

onSubtitleChange?: (event: SubtitleChangeEvent) => void

Called when subtitle text changes (new sentence/phrase).

onSubtitleHighlight​

onSubtitleHighlight?: (event: WordHighlightEvent) => void

Called for word-by-word highlighting (karaoke-style).

onImageChange​

onImageChange?: (event: ImageChangeEvent) => void

Called when a generated image is received.

onAvatarAnimationChanged​

onAvatarAnimationChanged?: (payload: { name: string; layer: number; loop: boolean }) => void

Called when avatar animation state changes.

onSafetyEvent​

onSafetyEvent?: (event: SafetyEvent) => void

Called when content triggers a safety policy.

onStringMessage​

onStringMessage?: (message: StringMessage) => void

Called for raw string messages from utilities.

Complete Example​

import { ConversationManager, ConversationConfig } from 'ug-js-sdk';

const config: ConversationConfig = {
  // Required
  apiUrl: 'wss://pug.stg.uglabs.app',
  apiKey: process.env.UG_API_KEY!,
  federatedId: `user-${userId}`,

  // Content
  prompt: `You are a helpful assistant for {{appName}}.
Be friendly and concise.`,
  context: {
    appName: 'My App',
  },

  // Capabilities
  capabilities: {
    audio: true,
    subtitles: true,
  },
  inputCapabilities: {
    audio: true,
    text: true,
  },

  // Voice
  voiceProfile: {
    provider: 'elevenlabs',
    voice_id: 'voice-id',
    speed: 1.0,
  },

  // Utilities
  utilities: {
    sentiment: {
      type: 'classify',
      classification_question: 'Sentiment of: {{user_input}}',
      answers: ['positive', 'negative', 'neutral'],
    },
  },
  onOutputUtilities: ['sentiment'],

  // Audio
  recordingConfig: {
    echoCancellation: true,
    noiseSuppression: true,
  },
  languageCode: 'en',

  // Hooks
  hooks: {
    onStateChange: (state) => {
      console.log('State:', state);
    },
    onTextMessage: (event) => {
      console.log('Text:', event.text);
    },
    onDataMessage: (event) => {
      console.log('Sentiment:', event.data.sentiment);
    },
    onError: (error) => {
      console.error('Error:', error.type, error.message);
    },
  },
};

const manager = new ConversationManager(config);
await manager.initialize();