Skip to main content
Use this when you have two agents for the same job, for example on two voice platforms or with two different prompts. Both run against the same Digital Humans and custom metrics, so the agent is the only difference between the runs.

​
Before you start

  • Connect both agents to Bluejay. See Connect your agent.
  • Create the custom metrics you want to compare the agents on. In each metric’s Agents field, select both agents. A metric that does not include an agent is not scored for it, so the comparison has nothing to show for that agent.
The Create Metric dialog with a Pass/Fail metric and two agents selected in the Agents field The Create Metric dialog with a Pass/Fail metric and two agents selected in the Agents field

​
Run both agents

1

Run agent A

On Simulations, create a simulation for the first agent with any simulation type. Under Custom Metric Settings, select the metrics you want to compare, then run it.The Custom Metric Settings tab of the Create Simulation dialog, with the Select all and Clear all buttons and the count of selected metricsThe Custom Metric Settings tab of the Create Simulation dialog, with the Select all and Clear all buttons and the count of selected metrics
2

Run agent B on the same Digital Humans

Create a second simulation for the other agent and select the Use Previous Digital Humans type. Pick the Digital Humans from agent A’s simulation, or the Community that holds them, so both agents face the same callers. Bluejay runs them as they are and does not generate new ones.Use Previous Digital Humans card in the Create Simulation type pickerUse Previous Digital Humans card in the Create Simulation type pickerUnder Custom Metric Settings, select the same metrics as for agent A, then run it. Different metrics make the runs impossible to compare.

​
Compare the runs

1

Stage the run from each simulation

Open the first simulation, select its run in Run History, and click Compare Runs. Do the same in the second simulation. Compare is unavailable until at least 2 runs are staged.Run History with one run checked and the Compare Runs button enabledRun History with one run checked and the Compare Runs button enabledThe tray at the bottom of the screen keeps staged runs while you move between simulations. A full page reload clears it, so select the runs again if that happens.
2

Open the comparison

Expand the tray to check that it holds one run per agent, then click Compare. Bluejay opens Compare Runs Across Agents.The expanded compare tray with one run from each of two agents and the Compare buttonThe expanded compare tray with one run from each of two agents and the Compare button
3

Read it one metric at a time

Set Compare by to one custom metric to see which agent fails which check. Start with Custom metrics passed (summary) to see which agent passes more overall. A metric with a free-text or JSON result does not appear in Compare by, because its results have no value to compare.
Last modified on October 22, 2026