Skip to main content

Utilities

Utilities allow you to extract structured information from user inputs or assistant outputs. There are two types of utilities: Classification and Extraction.

Overview​

Utilities are defined in the configuration and can be triggered:

  • on_input: Process user input before generating a response
  • on_output: Process assistant output after generation

Classification Utilities​

Classification utilities assign one of predefined categories to the input/output.

Structure​

{
  type: 'classify',
  classification_question: string,
  answers: string[]
}

Example: Sentiment Analysis​

{
  "sentiment": {
    "type": "classify",
    "classification_question": "What is the sentiment of this message: {{user_input}}",
    "answers": ["positive", "negative", "neutral"]
  }
}

When triggered, this will return:

{
  "kind": "data",
  "uid": "...",
  "data": {
    "sentiment": "positive"
  }
}

Example: Language Detection​

{
  "language": {
    "type": "classify",
    "classification_question": "What language is this text: {{user_input}}",
    "answers": ["English", "Spanish", "French", "German", "Chinese", "Other"]
  }
}

Template Variables​

  • {{user_input}}: The user's message
  • {{assistant_output}}: The assistant's response (only for on_output)
  • {{conversation_history}}: Recent conversation context

Extraction Utilities​

Extraction utilities extract specific information from the input/output as free-form text.

Structure​

{
  type: 'extract',
  extract_prompt: string
}

Example: Topic Extraction​

{
  "main_topic": {
    "type": "extract",
    "extract_prompt": "Extract the main topic from this text in 2-5 words: {{user_input}}"
  }
}

Returns:

{
  "kind": "data",
  "uid": "...",
  "data": {
    "main_topic": "Machine Learning"
  }
}

Example: Image Prompt Generation​

{
  "image_prompt": {
    "type": "extract",
    "extract_prompt": "Based on this story segment, create a detailed image generation prompt: {{assistant_output}}"
  }
}

Triggering Utilities​

On Input (Before Response)​

Specify utilities to run before generating a response:

await conversation.sendText('I love this product!', {
  on_input: ['sentiment', 'language']
});

The utilities will process the user input, and their results will be available before the assistant responds.

On Output (After Response)​

Specify utilities to run after generating a response:

await conversation.sendText('Tell me a story about a cat', {
  on_output: ['image_prompt']
});

The utility will process the assistant's output and return structured data.

Complete Example: Customer Support Bot​

await conversation.setConfiguration({
  prompt: 'You are a customer support assistant.',
  utilities: {
    issue_type: {
      type: 'classify',
      classification_question: 'What type of issue is this: {{user_input}}',
      answers: [
        'Technical Issue',
        'Billing Question',
        'Product Information',
        'Shipping & Delivery',
        'Returns & Refunds'
      ]
    },
    urgency: {
      type: 'classify',
      classification_question: 'What is the urgency: {{user_input}}',
      answers: ['Low', 'Medium', 'High', 'Critical']
    },
    ticket_summary: {
      type: 'extract',
      extract_prompt: 'Summarize this support request in one sentence: {{user_input}}'
    }
  }
});

// Listen for utility results
conversation.on('data-event', (event) => {
  console.log('Issue Type:', event.data.issue_type);
  console.log('Urgency:', event.data.urgency);
  console.log('Summary:', event.data.ticket_summary);
});

// Send message with utilities triggered on input
await conversation.sendText(
  'My order arrived damaged and I need a refund urgently!',
  { on_input: ['issue_type', 'urgency', 'ticket_summary'] }
);

Use Cases​

Sentiment Analysis​

Classify user messages by emotional tone:

{
  "sentiment": {
    "type": "classify",
    "classification_question": "Sentiment: {{user_input}}",
    "answers": ["positive", "negative", "neutral", "mixed"]
  }
}

Intent Recognition​

Determine what the user wants to do:

{
  "intent": {
    "type": "classify",
    "classification_question": "What does the user want: {{user_input}}",
    "answers": [
      "Get Information",
      "Make Purchase",
      "Get Support",
      "Cancel Service",
      "Other"
    ]
  }
}

Entity Extraction​

Extract specific information:

{
  "customer_name": {
    "type": "extract",
    "extract_prompt": "Extract the customer's name from: {{user_input}}"
  },
  "order_number": {
    "type": "extract",
    "extract_prompt": "Extract any order number mentioned in: {{user_input}}"
  }
}

Content Generation​

Generate structured content from assistant output:

{
  "email_subject": {
    "type": "extract",
    "extract_prompt": "Create an email subject line based on: {{assistant_output}}"
  },
  "key_points": {
    "type": "extract",
    "extract_prompt": "List 3 key points from: {{assistant_output}}"
  }
}

Image Generation Automation​

Automatically generate images for storytelling:

{
  "scene_description": {
    "type": "extract",
    "extract_prompt": "Create a detailed visual description of the scene in: {{assistant_output}}"
  }
}

Then use the extracted description to generate images:

conversation.on('data-event', async (event) => {
  if (event.data.scene_description) {
    await conversation.generateImage({
      prompt: event.data.scene_description,
      provider: 'replicate'
    });
  }
});

Best Practices​

  1. Keep classification answers mutually exclusive - Each answer should represent a distinct category
  2. Be specific in extract prompts - Clearly describe what you want to extract
  3. Limit the number of categories - 3-10 answers work best for classification
  4. Use descriptive variable names - Name utilities based on what they extract
  5. Test with various inputs - Ensure utilities work across different user messages
  6. Combine utilities - Use multiple utilities together for rich structured data

Limitations​

  • Classification utilities support up to 20 answer options
  • Extract prompts should be concise (under 500 characters)
  • Utilities add latency proportional to their complexity
  • Some utilities may not work well with very short inputs

Examples in the API Tester​

Try these example scenarios in the API Tester:

  • Sentiment Analysis - Classify message sentiment
  • Product Recommendation - Extract user preferences
  • Customer Support - Categorize support requests
  • Storytelling - Extract image prompts from story text

Next Steps​