> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getbluejay.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B Test Two Agents

> Compare two agents on the same callers.

Use this when you have two agents for the same job, for example on two voice platforms or with two different prompts. Both run against the same Digital Humans and custom metrics, so the agent is the only difference between the runs.

## Before you start

* Connect both agents to Bluejay. See [Connect your agent](/test/simulations/create.md#connect-your-agent).
* Create the [custom metrics](/test/metrics.md) you want to compare the agents on. In each metric's **Agents** field, select both agents. A metric that does not include an agent is not scored for it, so the comparison has nothing to show for that agent.

![The Create Metric dialog with a Pass/Fail metric and two agents selected in the Agents field](https://mintcdn.com/bluejay/UqiH6o220FskMm0H/images/cookbook-ab-test-metric-agents.png?fit=max\&auto=format\&n=UqiH6o220FskMm0H\&q=85\&s=49710dd352d2c4d977fbf3289a8bc1b2)

## Run both agents

**1. Run agent A**

On **Simulations**, create a simulation for the first agent with any [simulation type](/test/simulations/create.md#create-a-simulation). Under **Custom Metric Settings**, select the metrics you want to compare, then run it.

![The Custom Metric Settings tab of the Create Simulation dialog, with the Select all and Clear all buttons and the count of selected metrics](https://mintcdn.com/bluejay/UqiH6o220FskMm0H/images/cookbook-ab-test-agent-a-metrics.png?fit=max\&auto=format\&n=UqiH6o220FskMm0H\&q=85\&s=8bff508c2427f88bf39d05b40d56e86e)

**2. Run agent B on the same Digital Humans**

Create a second simulation for the other agent and select the **Use Previous Digital Humans** type. Pick the Digital Humans from agent A's simulation, or the Community that holds them, so both agents face the same callers. Bluejay runs them as they are and does not generate new ones.

![Use Previous Digital Humans card in the Create Simulation type picker](https://mintcdn.com/bluejay/EeUk0hD_moG9tWmI/images/create-simulation-card-previous-digital-humans.png?fit=max\&auto=format\&n=EeUk0hD_moG9tWmI\&q=85\&s=c6743e97118c240a2ce98ec21ab49500)

Under **Custom Metric Settings**, select the same metrics as for agent A, then run it. Different metrics make the runs impossible to compare.

## Compare the runs

**1. Stage the run from each simulation**

Open the first simulation, select its run in **Run History**, and click **Compare Runs**. Do the same in the second simulation. Compare is unavailable until at least 2 runs are staged.

![Run History with one run checked and the Compare Runs button enabled](https://mintcdn.com/bluejay/UqiH6o220FskMm0H/images/cookbook-ab-test-run-history.png?fit=max\&auto=format\&n=UqiH6o220FskMm0H\&q=85\&s=5d660bf48cfa3a6296dc141fe6ad810d)

The tray at the bottom of the screen keeps staged runs while you move between simulations. A full page reload clears it, so select the runs again if that happens.

**2. Open the comparison**

Expand the tray to check that it holds one run per agent, then click **Compare**. Bluejay opens **Compare Runs Across Agents**.

![The expanded compare tray with one run from each of two agents and the Compare button](https://mintcdn.com/bluejay/UqiH6o220FskMm0H/images/cookbook-ab-test-tray.png?fit=max\&auto=format\&n=UqiH6o220FskMm0H\&q=85\&s=3e2cf2ef1c0fb7f0b3777d69af7a0f53)

**3. Read it one metric at a time**

Set **Compare by** to one custom metric to see which agent fails which check. Start with **Custom metrics passed (summary)** to see which agent passes more overall. A metric with a free-text or JSON result does not appear in **Compare by**, because its results have no value to compare.

## Related

* [Previous Digital Humans](/test/simulation-types/previous-digital-humans.md)
* [Create a Simulation](/test/simulations/create.md)
* [Simulation Results](/test/simulations/create.md#read-the-results)
* [Digital Human Reference](/test/simulations/digital-humans.md)
* [Metrics](/test/metrics.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.