Skip to main content

ConversationManager

ug-js-sdk

ug-js-sdk / ConversationManager

Class: ConversationManager

Defined in: conversation-manager/ConversationManager.ts:119

ConversationManager is the main entry point for the UG Labs SDK. It manages real-time AI conversations with support for voice, text, and image generation.

The ConversationManager handles:

  • WebSocket connection to UG Labs API
  • Voice input and output (audio recording and playback)
  • Text-based interactions
  • Image generation via REST API
  • Conversation state management
  • Event-driven architecture for real-time updates

Connection Requirements​

Methods that DO NOT require an active WebSocket connection:

  • generateImage() - Uses REST API directly (generic, supports both providers)
  • generateImageWithBria() - Uses REST API directly (Bria-specific)
  • generateImageWithReplicate() - Uses REST API directly (Replicate-specific)
  • getAvailableModels() - Uses REST API directly
  • isConnected() - Check connection status
  • dispose() - Cleanup resources

Methods that REQUIRE an active WebSocket connection (call initialize() first):

  • initialize() - Establishes the connection
  • startListening() - Start voice input
  • stopListening() - Stop voice input
  • interact() - Send interactions
  • sendText() - Send text messages
  • interrupt() - Interrupt AI response
  • pause() - Pause playback
  • resume() - Resume playback
  • forceInputComplete() - Force end of user input
  • stop() - Stop and disconnect
  • toggleTextOnlyInput() - Toggle input mode

Examples​

import { ConversationManager } from 'ug-js-sdk';

const manager = new ConversationManager({
  apiKey: 'your-api-key',
  federatedId: 'user-123',
  prompt: 'You are a helpful assistant.'
});

// Listen for events
manager.on('text-event', (event) => {
  console.log('AI says:', event.text);
});

// Initialize and start conversation
await manager.initialize();
await manager.sendText('Hello!');
const manager = new ConversationManager({
  apiKey: 'your-api-key',
  inputCapabilities: { audio: true, text: true },
  capabilities: { audio: true, subtitles: true }
});

await manager.initialize();

// Start listening for voice input
await manager.startListening();
// User speaks...
await manager.stopListening();
// AI responds with voice
const manager = new ConversationManager({ apiKey: 'your-api-key' });

// Generate image without WebSocket connection
const imageUrl = await manager.generateImage({
  prompt: 'A serene mountain landscape',
  provider: 'bria'
});

Extends​

  • EventEmitter

Implements​

Constructors​

Constructor​

new ConversationManager(config): ConversationManager

Defined in: conversation-manager/ConversationManager.ts:164

Creates a new ConversationManager instance.

Parameters​

config​

ConversationConfig

Configuration for the conversation manager

Returns​

ConversationManager

Example​

const manager = new ConversationManager({
  apiKey: process.env.UG_API_KEY,
  federatedId: 'user-123',
  prompt: 'You are a helpful coding assistant.',
  utilities: {
    sentiment: {
      type: 'classify',
      classification_question: 'What is the sentiment?',
      answers: ['positive', 'negative', 'neutral']
    }
  }
});

Overrides​

EventEmitter.constructor

Properties​

isSimulateErrorsDebugMode​

isSimulateErrorsDebugMode: boolean = false

Defined in: conversation-manager/ConversationManager.ts:133

conversationCommands​

conversationCommands: ConversationCommands

Defined in: conversation-manager/ConversationManager.ts:134

logger​

protected logger: ILogger

Defined in: core/EventEmitter.ts:5

Inherited from​

AudioPlayer.logger

Methods​

setConfiguration()​

setConfiguration(config): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:210

Updates the configuration locally and propagates it to the backend server. If the network is not yet connected, only local config is updated.

Can be called with or without await:

  • Without await: Local config updates immediately, remote update is attempted (errors are logged)
  • With await: Can catch errors if remote update fails

Parameters​

config​

ConversationConfig

Returns​

Promise<void>

initialize()​

initialize(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:276

Initializes the conversation manager and establishes connections.

Requires: Network connection will be established during this call.

This method:

  • Authenticates with the API server
  • Initializes audio playback capabilities
  • Requests microphone access (if audio input is enabled)
  • Establishes WebSocket connection for real-time communication

Must be called before using conversation features like startListening(), sendText(), or interact().

Note: generateImage() and getAvailableModels() can be called without initialization.

Returns​

Promise<void>

Throws​

Will call onError hook with 'server_error' if initialization fails

Example​

const manager = new ConversationManager(config);
await manager.initialize();
// Now ready for conversation
await manager.startListening();

Implementation of​

IConversationManager.initialize

startListening()​

startListening(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:328

Starts listening for voice input from the user's microphone.

Requires: Active WebSocket connection (call initialize() first).

This method activates the Voice Activity Detection (VAD) system to detect when the user starts and stops speaking. Audio is captured and sent to the server for processing.

Returns​

Promise<void>

Throws​

Error if not in 'idle' state (must be idle to start listening)

Throws​

Will call onError hook with 'mic_denied' if microphone access fails

Example​

await manager.initialize();
await manager.startListening();
// User can now speak, audio will be captured

Implementation of​

IConversationManager.startListening

stopListening()​

stopListening(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:354

Stops listening for voice input.

Requires: Active WebSocket connection.

Stops the microphone capture and VAD processing. The state will transition to 'waiting' if successfully stopped.

Returns​

Promise<void>

Example​

await manager.stopListening();

Implementation of​

IConversationManager.stopListening

sendText()​

sendText(text): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:379

Sends a text message to the conversation.

Requires: Active WebSocket connection (call initialize() first).

This is a convenience method that wraps interact() for simple text input. The message will be processed by the AI and a response will be generated.

Parameters​

text​

string

The text message to send

Returns​

Promise<void>

Example​

await manager.initialize();
await manager.sendText('Hello, how are you?');
// Response will come through onTextMessage hook

Implementation of​

IConversationManager.sendText

generateImage()​

generateImage(params): Promise<any>

Defined in: conversation-manager/ConversationManager.ts:424

Generates an image using the specified provider (Bria or Replicate).

Note: This method does NOT require an active WebSocket connection. It uses REST API endpoints directly, so it can be called before initialize() or even if the WebSocket connection is disconnected.

Parameters​

params​

GenerateImageParams

Image generation parameters

Returns​

Promise<any>

Promise resolving to the generated image response containing the Base64 image

Example​

// Generate image without needing an active connection
const result = await conversationManager.generateImage({
  prompt: 'A beautiful sunset over mountains',
  provider: 'bria'
});
console.log(result.image); // Base64 encoded image

Implementation of​

IConversationManager.generateImage

generateImageWithBria()​

generateImageWithBria(params): Promise<any>

Defined in: conversation-manager/ConversationManager.ts:459

Generates an image using Bria.

Note: This method does NOT require an active WebSocket connection. It uses REST API endpoints directly, so it can be called before initialize() or even if the WebSocket connection is disconnected.

Parameters​

params​

GenerateImageBriaParams

Bria-specific image generation parameters

Returns​

Promise<any>

Promise resolving to the generated image response containing the Base64 image

Example​

// Generate image with Bria without needing an active connection
const result = await conversationManager.generateImageWithBria({
  prompt: 'A beautiful sunset over mountains',
  generation_type: 'fast'
});
console.log(result.image); // Base64 encoded image

Implementation of​

IConversationManager.generateImageWithBria

generateImageWithReplicate()​

generateImageWithReplicate(params): Promise<any>

Defined in: conversation-manager/ConversationManager.ts:495

Generates an image using Replicate.

Note: This method does NOT require an active WebSocket connection. It uses REST API endpoints directly, so it can be called before initialize() or even if the WebSocket connection is disconnected.

Parameters​

params​

GenerateImageReplicateParams

Replicate-specific image generation parameters

Returns​

Promise<any>

Promise resolving to the generated image response containing the Base64 image

Example​

// Generate image with Replicate without needing an active connection
const result = await conversationManager.generateImageWithReplicate({
  prompt: 'A cyberpunk cityscape at night',
  model: 'black-forest-labs/flux-schnell-lora'
});
console.log(result.image); // Base64 encoded image

Implementation of​

IConversationManager.generateImageWithReplicate

getAvailableModels()​

getAvailableModels(): Promise<any>

Defined in: conversation-manager/ConversationManager.ts:515

Retrieves the list of available image generation models.

Note: This method does NOT require an active WebSocket connection. It uses REST API endpoints directly, so it can be called before initialize() or even if the WebSocket connection is disconnected.

Returns​

Promise<any>

Promise resolving to an object containing available models as key-value pairs

Example​

// Get available models without needing an active connection
const result = await conversationManager.getAvailableModels();
console.log(result.models); // { "model-id": "Model Name", ... }

Implementation of​

IConversationManager.getAvailableModels

interact()​

interact(request): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:547

Sends an interaction request to the conversation.

Requires: Active WebSocket connection (call initialize() first).

This is the main method for sending messages to the AI. It handles both text and audio-based interactions. When called, it will:

  • Reset any ongoing playback
  • Send the request to the server
  • Trigger response processing via hooks (onTextMessage, onDataMessage, etc.)

Parameters​

request​

InteractRequest

The interaction request containing text, context, and options

Returns​

Promise<void>

Example​

await manager.interact({
  text: 'Tell me about the weather',
  context: { location: 'New York' },
  kind: 'interact',
  type: 'stream',
  uid: ''
});

Implementation of​

IConversationManager.interact

interrupt()​

interrupt(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:592

Interrupts the current AI response playback.

Requires: Active WebSocket connection.

Immediately stops any audio/text being played back and transitions to 'interrupted' state. Use this when you want to stop the AI mid-response.

Returns​

Promise<void>

Example​

// User wants to interrupt the AI while it's speaking
await manager.interrupt();

Implementation of​

IConversationManager.interrupt

pause()​

pause(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:613

Pauses the current AI response playback.

Requires: Active WebSocket connection and 'playing' state.

Pauses audio/text playback. Can be resumed later with resume(). Only works when in 'playing' state.

Returns​

Promise<void>

Example​

// Pause playback
await manager.pause();
// Later, resume it
await manager.resume();

Implementation of​

IConversationManager.pause

resume()​

resume(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:671

Resumes paused AI response playback.

Requires: Active WebSocket connection and 'paused' state.

Continues audio/text playback from where it was paused. Only works when in 'paused' state.

Returns​

Promise<void>

Example​

await manager.pause();
// ... some time later ...
await manager.resume();

Implementation of​

IConversationManager.resume

forceInputComplete()​

forceInputComplete(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:742

Forces the completion of user input.

Requires: Active WebSocket connection and 'userSpeaking' or 'listening' state.

Manually signals that the user has finished speaking, bypassing the automatic Voice Activity Detection (VAD) silence detection. Useful when VAD doesn't detect the end of speech properly.

Returns​

Promise<void>

Example​

// Force end of user input even if VAD hasn't detected silence
await manager.forceInputComplete();

Implementation of​

IConversationManager.forceInputComplete

stop()​

stop(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:766

Stops the conversation and disconnects from the server.

Requires: Active WebSocket connection.

Stops all audio recording, pauses playback, and closes the WebSocket connection. The state transitions to 'idle'. To restart, call initialize() again.

Note: For complete cleanup including resource release, use dispose() instead.

Returns​

Promise<void>

Example​

await manager.stop();
// Connection is closed, must re-initialize to continue

Implementation of​

IConversationManager.stop

dispose()​

dispose(): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:797

Disposes of all resources and performs complete cleanup.

Does NOT require an active connection - safe to call anytime.

This is the recommended method for cleanup when the conversation ends or when navigating away from the page. It performs a complete teardown:

  • Stops audio recording and releases the media stream (microphone)
  • Stops Voice Activity Detection (VAD) analysis
  • Disposes the audio recorder
  • Disposes the playback manager (closes AudioContext)
  • Closes the WebSocket connection
  • Removes all event listeners from this and child components

After calling dispose(), the ConversationManager instance should not be reused. Create a new instance if you need to start a new conversation.

Returns​

Promise<void>

Example​

// When done with the conversation or navigating away
await manager.dispose();
// manager is now cleaned up and should not be reused

Implementation of​

IConversationManager.dispose

toggleTextOnlyInput()​

toggleTextOnlyInput(isTextOnly): Promise<void>

Defined in: conversation-manager/ConversationManager.ts:1108

Toggles between text-only and audio input modes.

Requires: Active WebSocket connection for full functionality.

When text-only mode is enabled:

  • Microphone recording is stopped
  • Only text input via sendText() is accepted

When text-only mode is disabled:

  • Audio recording can be started
  • Voice input via startListening() is available

Parameters​

isTextOnly​

boolean

True to enable text-only mode, false to enable audio input

Returns​

Promise<void>

Example​

// Switch to text-only mode (no microphone)
await manager.toggleTextOnlyInput(true);
await manager.sendText('Hello');

// Switch back to audio mode
await manager.toggleTextOnlyInput(false);
await manager.startListening();

Implementation of​

IConversationManager.toggleTextOnlyInput

isConnected()​

isConnected(): boolean

Defined in: conversation-manager/ConversationManager.ts:1132

Checks if the WebSocket connection is currently active and ready.

Does NOT require initialization - can be called anytime.

Use this to check connection status before performing operations that require an active connection.

Returns​

boolean

True if connected and ready, false otherwise

Example​

if (manager.isConnected()) {
  await manager.sendText('Hello');
} else {
  console.log('Not connected, please initialize first');
}

Implementation of​

IConversationManager.isConnected

enableAudioDebugMode()​

enableAudioDebugMode(): void

Defined in: conversation-manager/ConversationManager.ts:1151

Enables audio debug mode for troubleshooting audio capture issues.

Does NOT require an active connection.

When enabled, audio chunks are captured and stored for later analysis. Use downloadCapturedAudio() to save the captured audio.

Returns​

void

Example​

manager.enableAudioDebugMode();
// ... capture audio ...
manager.downloadCapturedAudio('debug-audio.wav');

disableAudioDebugMode()​

disableAudioDebugMode(): void

Defined in: conversation-manager/ConversationManager.ts:1163

Disables audio debug mode.

Does NOT require an active connection.

Stops capturing audio chunks for debugging purposes.

Returns​

void

downloadCapturedAudio()​

downloadCapturedAudio(filename?): void

Defined in: conversation-manager/ConversationManager.ts:1186

Downloads captured audio from debug mode as a file.

Does NOT require an active connection.

Only works if enableAudioDebugMode() was called and audio was captured.

Parameters​

filename?​

string

Optional filename for the downloaded file

Returns​

void

Example​

manager.enableAudioDebugMode();
await manager.startListening();
// ... speak into microphone ...
await manager.stopListening();
manager.downloadCapturedAudio('my-recording.wav');

clearCapturedAudio()​

clearCapturedAudio(): void

Defined in: conversation-manager/ConversationManager.ts:1197

Clears all captured audio from debug mode.

Does NOT require an active connection.

Removes all stored audio chunks from memory.

Returns​

void

getCapturedAudioInfo()​

getCapturedAudioInfo(): object

Defined in: conversation-manager/ConversationManager.ts:1214

Gets information about captured audio in debug mode.

Does NOT require an active connection.

Returns​

object

Object containing chunk count and total size in bytes

chunkCount​

chunkCount: number

totalSize​

totalSize: number

Example​

const info = manager.getCapturedAudioInfo();
console.log(`Captured ${info.chunkCount} chunks, ${info.totalSize} bytes`);

on()​

on<K>(event, callback): void

Defined in: core/EventEmitter.ts:11

Subscribes to an event emitted by the conversation manager.

Type Parameters​

K​

K extends string

Parameters​

event​

K

callback​

any

Returns​

void

Implementation of​

IConversationManager.on

Inherited from​

EventEmitter.on

off()​

off<K>(event, callback): void

Defined in: core/EventEmitter.ts:18

Type Parameters​

K​

K extends string

Parameters​

event​

K

callback​

any

Returns​

void

Inherited from​

EventEmitter.off

emit()​

emit<K>(event, data?): Promise<void>

Defined in: core/EventEmitter.ts:28

Type Parameters​

K​

K extends string

Parameters​

event​

K

data?​

unknown

Returns​

Promise<void>

Inherited from​

EventEmitter.emit

removeAllListeners()​

removeAllListeners(event?): void

Defined in: core/EventEmitter.ts:73

Parameters​

event?​

string

Returns​

void

Inherited from​

EventEmitter.removeAllListeners