Skip to main content
In Bluejay, an Agent is the voice or chat system you are evaluating. Every simulation, observability run, custom metric, and workflow attaches to a specific Agent, which gives Bluejay the context it needs to generate realistic conversations and score the agent against the behaviors you care about.

​
What Is an Agent?

An Agent is the canonical object that represents a single voice or chat system under evaluation. Once you create an Agent, every downstream feature in Bluejay is scoped to that Agent. You can have many Agents in one Bluejay account, typically one per system you want to test or monitor. Each Agent carries its own configuration, knowledge base, and evaluation history.

​
Anatomy of an Agent

An Agent is made up of a small set of structured fields:

​
Creating an Agent

You can create an Agent from the Bluejay dashboard or from the API.
  • Dashboard. Click Create Agent, pick the agent type (Inbound or Outbound), mode (Voice or Text), and connection type, then fill in the connection details.
  • API. Send a POST to the Add Agent endpoint with the same fields in a JSON body.

​
Connecting an Agent to a Provider

To finish wiring an Agent up for live testing or observability, follow the provider-specific setup guide: Each guide walks through configuring the provider webhook, saving credentials inside Bluejay, and binding a Bluejay Agent to the provider Agent ID.

​
How Agents Connect to the Rest of Bluejay

  • Simulations dial out (or send chat messages) to the Agent through its provider connection, then score the resulting conversation against your custom metrics and goals.
  • Observability receives webhook events from production calls and matches them to the Agent. Each call is stored as a run you can review, score, and aggregate.
  • Custom metrics are defined per Agent and execute against every simulation result and every observability event.
  • Workflows (for ElevenLabs, Vapi, Retell, and Google Dialogflow CX agents) sync into Bluejay and render as an editable graph on the Agent’s Workflow tab.

​
Typical Agent Lifecycle

  1. Create the Agent in Bluejay (in the dashboard or through the Add Agent API).
  2. Configure the Agent by setting goals and attaching a knowledge base.
  3. Connect the Agent to its production system by choosing the connection type and entering the phone number or provider agent ID.
  4. Test by running simulations against the Agent to verify behavior before changes ship.
  5. Monitor by turning on observability so production calls flow into Bluejay for ongoing scoring and alerting.
  6. Iterate by updating goals, knowledge base content, or provider configuration as the agent evolves.

​
Best Practices

  • Be specific. Clear, detailed descriptions and goals make simulations and analytics highly relevant and actionable. Vague goals lead to vague scoring.
  • Keep the knowledge base current. Update FAQs, scripts, and guidelines whenever the agent’s intent or content changes, so simulations continue to reflect production behavior.
  • Use distinct names. Intuitive, descriptive names make it easier to identify the right Agent in the dashboard, in API responses, and in webhook payloads.
  • Bind the right phone number. Ensure the phone number on the Agent matches the production number, so observability events flow to the correct Agent and metrics aggregate correctly.

​
Example Agent Definition

​
Resources

Workflows

Sync multi-agent flows from your provider and use them to drive simulations.

Add Agent API

Create an Agent programmatically.

Simulations Overview

How simulations work in Bluejay, from setup to scoring.

Custom Metrics

Define scoring rules that apply to every simulation and every production call for an Agent.