Skip to main content
Use Metrics Lab to check a metric’s verdicts against your own and improve its prompt. You label conversations with the verdict you expect, then test prompt revisions until the metric agrees with you.

​
Build a lab

A lab pairs one or more custom metrics with a sample of conversations from an agent or a simulation run. You test prompt revisions against this sample, so the result is only as good as the sample. Select 20 to 50 conversations for the lab, and include clear passes, clear fails, and unclear cases. The Create Lab dialog with the name, metrics, and conversations steps The Create Lab dialog with the name, metrics, and conversations steps

​
Label the conversations

Your label is the human verdict, and the metric’s result is the AI verdict. The alignment score shows how often the two agree. To save time, label in bulk when many rows share the same verdict. The Annotate tab with AI and Human columns The Annotate tab with AI and Human columns
Label every conversation before you read the alignment score. The score compares only rows that have a human verdict.
To test prompt revisions and publish the best one, see Improve the accuracy of a metric.
Last modified on October 26, 2026