LiveKit Observability

This guide walks you through how to connect your production LiveKit agents to Bluejay’s observability software. Once connected, every call shows up in your logs view shortly after it ends.

Tutorial

API
Web (No Code)

This integration method enables Bluejay to observe every call per agent. Your agent POSTs the finished call to Bluejay’s /evaluate endpoint when the session ends.

If you already configured the Web (No Code) webhook in the same LiveKit project as the agent, set the bluejay.bypass_webhook attribute to true when the agent joins. Otherwise, the call will be ingested twice.

await req.accept(identity="support-bot", attributes={"bluejay.bypass_webhook": "true"}, ...)

Integration Prompt for AI Agents

# Bluejay — Testing & Monitoring Platform for Conversational AI Agents

You are a senior backend engineer integrating the Bluejay API. Think step-by-step: first understand the endpoint, then plan the integration, then implement with minimal changes.

## LiveKit Observability — POST /v1/evaluate

> **What this endpoint does:** Report a finished LiveKit call (transcript, tool calls, and optionally a recording) to Bluejay from your agent's shutdown callback, so it appears in the logs view with evals.

**Endpoint:** POST `https://api.getbluejay.ai/v1/evaluate`
**Auth:** `X-API-Key` header
**Content-Type:** application/json

### Required Parameters
| Name | Type | Description |
|------|------|-------------|
| agent_id | string | Existing Bluejay agent to attribute the call to (the endpoint does not auto-create it). |
| start_time_utc | string | Call start, ISO 8601 UTC, captured at call start. |
| participants | array | Call roles, e.g. `[{"role": "AGENT"}, {"role": "USER"}]`. |
| transcript / recording_url | array / string | Provide at least one: the transcript turns and/or a recording URL (a recording also unlocks audio-derived metrics). |
| X-API-Key | string | API key required to authenticate requests. |

Review the full parameter list at https://docs.getbluejay.ai/api-reference/endpoint/evaluate and include any optional parameters (e.g., `tool_calls`, `recording_url`, `metadata`, `label`) that serve your integration's use case and align with Bluejay's testing and monitoring capabilities.

### Request Body
```json
{
  "agent_id": "<your-bluejay-agent-id>",
  "start_time_utc": "2026-01-01T00:00:00Z",
  "participants": [{"role": "AGENT"}, {"role": "USER"}],
  "transcript": [
    {"start_offset_ms": 0, "end_offset_ms": 1500, "speaker": "AGENT", "utterance": "Hi, how can I help?"}
  ],
  "tool_calls": [
    {"name": "lookup_order", "start_offset_ms": 4200, "parameters": {"id": "123"}, "output": "shipped"}
  ],
  "recording_url": "https://...optional..."
}
```

### Example
**Register a shutdown callback that assembles the call and POSTs it:**
```python
import os, httpx
from livekit.agents import JobContext

async def entrypoint(ctx: JobContext):
    # ... agent setup + session.start(...) ...
    async def report_to_bluejay():
        payload = {
            "agent_id": os.environ["BLUEJAY_AGENT_ID"],
            "start_time_utc": started_at_iso,  # ISO 8601, UTC, captured at call start
            "participants": [{"role": "AGENT"}, {"role": "USER"}],
            "transcript": transcript,   # [{start_offset_ms,end_offset_ms,speaker,utterance}, ...]
            "tool_calls": tool_calls,   # [{name,parameters,output,start_offset_ms}, ...]
        }
        async with httpx.AsyncClient(timeout=10) as client:
            r = await client.post(
                "https://api.getbluejay.ai/v1/evaluate",
                json=payload,
                headers={"X-API-Key": os.environ["BLUEJAY_API_KEY"]},
            )
            r.raise_for_status()
    ctx.add_shutdown_callback(report_to_bluejay)
```

### Constraints
- Minimal changes — only add/change files needed for this integration.
- Match existing codebase patterns (naming, file structure, error handling).
- Capture `start_time_utc` at call start (ISO 8601, UTC), and handle a 422 Validation Error from the endpoint.
- If this LiveKit project also has the Bluejay Cloud webhook configured, set `attributes={"bluejay.bypass_webhook": "true"}` on `req.accept(...)` so the call isn't ingested twice.

### Integration Checklist
Before writing code, verify:
1. Where does the agent build/track the transcript and tool calls during the session?
2. What HTTP client and error-handling patterns does the project use?
3. Are the Bluejay API key and agent id available as config/env?

Then implement the shutdown callback, export it, and confirm it compiles/passes lint.

Prerequisites

A Bluejay API key (Settings → API Keys)
Access to your agent’s code

Create a Bluejay agent

The /evaluate endpoint attributes each call to an existing Bluejay agent, so create one first.

In Bluejay, create an agent (or open an existing one) and copy its Agent ID.
In the agent’s Connection Settings tab, choose the LiveKit tile under Voice connection type.

POST the finished call at shutdown

Register a shutdown callback that assembles the call and POSTs it to Bluejay. Include tool_calls for tool-adherence metrics, and recording_url for audio-derived metrics.

import os
import httpx
from livekit.agents import JobContext

# ... your code ...

async def entrypoint(ctx: JobContext):
    # ... your code ...

    async def report_to_bluejay():
        payload = {
            "agent_id": "<your-bluejay-agent-id>",
            "start_time_utc": started_at_iso,  # ISO 8601, UTC, captured at call start
            "participants": [{"role": "AGENT"}, {"role": "USER"}],
            # [{"start_offset_ms", "end_offset_ms", "speaker", "utterance"}, ...]
            "transcript": transcript,
            # [{"name", "parameters", "output", "start_offset_ms"}, ...]
            "tool_calls": tool_calls,
            # Optional — audio-derived metrics (latency, talk ratio, WER):
            # "recording_url": recording_url,
        }
        async with httpx.AsyncClient(timeout=10) as client:
            res = await client.post(
                "https://api.getbluejay.ai/v1/evaluate",
                json=payload,
                headers={"X-API-Key": os.environ["BLUEJAY_API_KEY"]},
            )
            res.raise_for_status()

    ctx.add_shutdown_callback(report_to_bluejay)

When the call ends, the callback submits it and the evaluated call appears in your logs view. See the /evaluate API reference for the full request shape.

This integration enables Bluejay to observe every call for all agents in the project, using a webhook.

This integration method cannot observe tool calls, and may incur additional costs from LiveKit for egress.

Prerequisites

A LiveKit Cloud project (API Key, API Secret, and Server URL)
A Bluejay account

Get your LiveKit credentials

In the LiveKit Cloud dashboard, open your project and go to Settings → API keys.
Click Create Key → Generate key, then copy your API Key, API Secret, and Server URL.

Add your LiveKit credentials in Bluejay

In Bluejay, click Settings in the bottom left corner.
Select Integrations and open the LiveKit tab.
Enter your API Key, API Secret, and URL from LiveKit, then click Save.
Click Show configuration and copy your webhook URL.

Add the webhook in LiveKit

In the LiveKit Cloud dashboard, go to Settings → Webhooks.
Add the webhook URL you copied, signed with your API Key.

Confirm your agent has a name

Your agent runs under a dispatch name in LiveKit. Bluejay will automatically create an agent under this name.Most agents already have one, and on the first call Bluejay automatically creates a matching agent for you. In most cases, there’s nothing to do here.

Need to set or change the name? (for developers)

In your agent code, set agent_name in your worker’s WorkerOptions. Then redeploy:

from livekit.agents import WorkerOptions, cli

cli.run_app(
    WorkerOptions(
        agent_name="set-the-name-here",
    )
)

Up Next

LiveKit Simulations

Run Bluejay simulations against your LiveKit agents.

Getting Started

Key Concepts

Test

Monitor

Integrations

Tutorial

Up Next

LiveKit Simulations

​Tutorial

​Up Next

LiveKit Simulations

Tutorial

Up Next