Dynamic Variables

Dynamic variables let you inject call-specific context directly into your Custom Metric definitions at evaluation time. Instead of writing a static metric that applies the same way to every conversation, you can write a metric template with {{placeholders}} that get filled in with real data from each individual call. This is especially powerful when the right answer depends on information that’s unique to each conversation, such as the customer’s name, their account tier, the product they called about, or any other runtime value your agent has access to.

How It Works

Dynamic variables use a simple {{key}} syntax in your metric’s description or scoring_guidance. When Bluejay evaluates a conversation, it replaces each {{key}} with the corresponding value from the metadata object you pass in the /v1/evaluate request.

Metric description:  "Did the agent greet {{customer_name}} by name and confirm the
                      account ending in {{account_last_four}} before proceeding?"

metadata in request: { "customer_name": "Sarah", "account_last_four": "4821" }

Evaluated as:        "Did the agent greet Sarah by name and confirm the account
                      ending in 4821 before proceeding?"

Setting Up a Dynamic Variable Metric

Write your metric with placeholders

Use {{variable_name}} anywhere in the description or scoring_guidance of your Custom Metric:

{
  "name": "Correct Product Quoted",
  "description": "Did the agent correctly describe the {{product_name}} plan, including the price of ${{monthly_price}} per month?",
  "response_type": "pass_fail"
}

Pass values in the evaluate request

When you submit a call for evaluation, include the matching key-value pairs in the top-level metadata field of the request body:

{
  "agent_id": "your-agent-id",
  "transcript": [...],
  "metadata": {
    "product_name": "Business Pro",
    "monthly_price": "149"
  }
}

Bluejay substitutes and evaluates

At evaluation time, Bluejay substitutes the placeholders before sending the prompt to the LLM judge. The variable replacement is automatic. No extra configuration is required.

Use Cases

Customer-specific assertions

Reference the customer’s name, account number, or tier directly in your metric criteria so the evaluation is grounded in the actual call context.

Product & pricing accuracy

Inject the specific product, plan, or price the customer called about so the metric can verify the agent quoted the right figures.

Compliance with call reason

Pass the stated reason for the call so compliance metrics can check whether the agent followed the correct protocol for that specific scenario.

Dynamic thresholds

Vary scoring expectations based on call type, customer segment, or business unit. Use a single metric template across many different contexts.

Example: Account Verification

Here’s a complete example combining a metric template with the corresponding evaluate request. Metric definition

{
  "name": "Account Verification Passed",
  "description": "The customer called in about their account ending in {{account_last_four}}. Did the agent verify the customer's identity by confirming at least two of: full name, date of birth, or billing zip code before discussing account details?",
  "response_type": "pass_fail",
  "scoring_guidance": "A pass requires the agent to confirm two verification factors before mentioning any account-specific information. If the agent only confirmed one factor or skipped verification entirely, this is a fail."
}

Evaluate request

{
  "agent_id": "your-agent-id",
  "transcript": [...],
  "metadata": {
    "account_last_four": "7732"
  }
}

Result. Bluejay evaluates the metric as:

“The customer called in about their account ending in 7732. Did the agent verify the customer’s identity…”

Tips & Gotchas

If a placeholder key is present in your metric description but not found in metadata, Bluejay leaves the {{key}} text as-is rather than failing the evaluation. Always verify your metadata keys match the placeholders exactly.

Key names are case-sensitive

{{Customer_Name}} and {{customer_name}} are treated as different keys. Match the casing in your metric exactly to the casing in your metadata payload.

Values are coerced to strings

Numbers and booleans work fine as metadata values. Bluejay converts them to strings before substituting into the prompt.

Placeholders work in description and scoring_guidance

Both fields are substituted before evaluation, so you can split context (description) from grading rules (scoring_guidance) and still reference the same dynamic values in each.

Metadata is scoped to the evaluation

You do not need to declare metadata anywhere on the metric itself. Provide the keys you want substituted at call time, and Bluejay uses them only for that evaluation.

Extra metadata keys are fine

Any keys that do not match a placeholder are simply stored as call context. They remain accessible in the call trace for debugging.

Resources

Prompting Guide

Write the surrounding metric prompt that the variables plug into.

Metric Types

Pick the right response type for each measurement goal.

Metrics Lab

Test dynamic variable metrics against sample transcripts before going live.

Evaluate Endpoint

Submit calls and pass metadata for variable substitution.

Getting Started

Key Concepts

Test

Monitor

Integrations

How It Works

Setting Up a Dynamic Variable Metric

Use Cases

Customer-specific assertions

Product & pricing accuracy

Compliance with call reason

Dynamic thresholds

Example: Account Verification

Tips & Gotchas

Resources

Prompting Guide

Metric Types

Metrics Lab

Evaluate Endpoint

​How It Works

​Setting Up a Dynamic Variable Metric

​Use Cases

Customer-specific assertions

Product & pricing accuracy

Compliance with call reason

Dynamic thresholds

​Example: Account Verification

​Tips & Gotchas

​Resources

Prompting Guide

Metric Types

Metrics Lab

Evaluate Endpoint

How It Works

Setting Up a Dynamic Variable Metric

Use Cases

Example: Account Verification

Tips & Gotchas

Resources