Skip to main content
Use an alert with a metric to catch failures on your production conversations. To create alerts from code, use the Alerts API. The Alerts list with one row for each alert, showing its status such as In Alert or Active, last triggered time, metric, condition, threshold interval, threshold events, and scope The Alerts list with one row for each alert, showing its status such as In Alert or Active, last triggered time, metric, condition, threshold interval, threshold events, and scope

​
Connect a channel

  1. In Bluejay, open Settings → Integrations and choose the Slack tab. Connect Slack there.
  2. In each alert, pick the Slack channel it posts to.

​
Create an alert

1

Open the dialog

In the sidebar under Observability, open Alerts and click Create Alert.
2

Set the trigger and scope

For a metric where one failure is too many, such as hallucination, set for at least to 1.An alert watches only its own workspace, even when set to all agents, so create it in the workspace of the agents it should watch.
3

Choose who gets notified

Send failures that need action now to PagerDuty or Better Stack, and quality drops to Slack or email. Only connected channels can be turned on.Create Alert dialog with Name, Trigger, Scope, and Notify sections, Slack Notifications on, and a Slack preview with Send test notificationCreate Alert dialog with Name, Trigger, Scope, and Notify sections, Slack Notifications on, and a Slack preview with Send test notification

​
How alerts evaluate

An alert counts the production conversations that match its condition in the window. It enters alarm when at least N conversations match, where N is the for at least number, and recovers when the count drops below 80% of N (minimum 1). With a window such as the last 50 conversations, the alert does not fire until that many conversations exist.

​
Incidents

PagerDuty and Better Stack receive incidents, and your escalation policy in that tool decides who gets the page.
Last modified on October 21, 2026