> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getbluejay.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# CI/CD Integration

> Fail a build when your agent's pass rate drops.

Add the Bluejay action to a GitHub workflow to run a simulation from CI.

## Prerequisites

1. A Bluejay [API key](/api-reference/introduction.md#make-your-first-request).
2. An agent connected to your assistant.
3. A simulation on that agent, with the Digital Humans you want to run.

However your agent connects to Bluejay, you only pass `api_key` and `simulation_id` to the CI workflow.

## Test an existing agent

Use this option unless CI makes a preview deployment for each PR. All runs go to the same simulation, so you can compare them.

**1. Add your API key and variables**

In GitHub, under **Settings > Secrets and variables > Actions**, add your API key as a repository secret named `BLUEJAY_API_KEY`, and add these repository variables:

| Variable name | Value | Required |
| - | - | - |
| BLUEJAY\_SIMULATION\_ID | Your simulation ID (e.g., `12345`) | Yes |
| BLUEJAY\_MIN\_SCORE | Minimum passing score (e.g., `80`) | No |
| BLUEJAY\_PROMPT\_ID | Agent version override ID (the action input is `prompt_id`) | No |
| BLUEJAY\_KB\_ID | Knowledge Base override ID | No |
| BLUEJAY\_DIGITAL\_HUMAN\_IDS | Comma-separated Digital Human IDs | No |

The simulation ID is in the simulation's **General Settings** tab, and each Digital Human ID is on its card in the **Digital Humans** tab.

**2. Create your workflow**

Add `.github/workflows/bluejay-tests.yml` to your repo:

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
name: Agent Tests

on:
  workflow_dispatch:
    inputs:
      simulation_id:
        description: 'Bluejay Simulation ID to run'
        required: false
        type: string
      min_score:
        description: 'Minimum required score (0–100)'
        required: false
        type: string
        default: '80'
  push:
    branches: [main]
  pull_request:
    types: [opened, synchronize]

jobs:
  run-bluejay-tests:
    runs-on: ubuntu-latest

    steps:
      - name: Run Bluejay Tests
        uses: bluejay-ai-dev/bluejay-github-actions@v1
        with:
          api_key: ${{ secrets.BLUEJAY_API_KEY }}
          # Required: simulation id (manual input OR repo variable)
          simulation_id: ${{ inputs.simulation_id || vars.BLUEJAY_SIMULATION_ID }}
          # Optional overrides (omit the ones you do not use)
          prompt_id: ${{ vars.BLUEJAY_PROMPT_ID }}
          knowledge_base_id: ${{ vars.BLUEJAY_KB_ID }}
          digital_human_ids: ${{ vars.BLUEJAY_DIGITAL_HUMAN_IDS }}
          # Behavior controls
          wait_for_results: 'true'
          min_score: ${{ inputs.min_score || vars.BLUEJAY_MIN_SCORE || '80' }}
          poll_interval_seconds: '10'
          timeout_seconds: '1500'
```

**3. Trigger a simulation**

Open a pull request. The action runs the simulation on each PR and shows the status and score in the PR checks.

![GitHub Action Check](https://mintcdn.com/bluejay/13QTZFHWYyYMSQI8/images/trigger_github_action.png?fit=max\&auto=format\&n=13QTZFHWYyYMSQI8\&q=85\&s=7fe8d0c5156098b39f9dfb33ed280524)

## Test a preview deployment

Use this option if CI makes a preview deployment for each PR. For each PR, the workflow creates an agent and a simulation, runs the test, and then deletes both. Your Digital Humans are not deleted, because they are your test suite and every PR reuses them.

In the `add-agent` payload, replace `<your connection field>` with the field that tells Bluejay how to reach your preview deployment. [Add Agent](/api-reference/endpoint/add-agent.md) lists the connection fields.

For this option, set `BLUEJAY_DIGITAL_HUMAN_IDS`, for example `101,102`, because a new simulation has no Digital Humans and the run fails without them.

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
name: Agent Tests (ephemeral)

on:
  pull_request:
    types: [opened, synchronize]

env:
  BLUEJAY_API: https://api.getbluejay.ai/v1
  BLUEJAY_API_KEY: ${{ secrets.BLUEJAY_API_KEY }}

jobs:
  run-bluejay-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Create ephemeral agent
        id: agent
        run: |
          payload=$(jq -n \
            --arg name "ci-pr-${{ github.event.number }}-run-${{ github.run_id }}" \
            --arg connection "<how Bluejay reaches this PR's preview deployment>" \
            '{
              name: $name,
              system_prompt: "Customer support assistant under test",
              goals: ["Resolve the customer issue"],
              "<your connection field>": $connection
            }')
          resp=$(curl -s --fail-with-body -X POST "$BLUEJAY_API/add-agent" \
            -H "X-API-Key: $BLUEJAY_API_KEY" \
            -H "Content-Type: application/json" \
            -d "$payload")
          echo "id=$(echo "$resp" | jq -r '.agent_id')" >> "$GITHUB_OUTPUT"

      - name: Create simulation
        id: sim
        run: |
          resp=$(curl -s --fail-with-body -X POST "$BLUEJAY_API/create-simulation" \
            -H "X-API-Key: $BLUEJAY_API_KEY" \
            -H "Content-Type: application/json" \
            -d '{
              "agent_id": "${{ steps.agent.outputs.id }}",
              "name": "CI PR #${{ github.event.number }}"
            }')
          echo "id=$(echo "$resp" | jq -r '.simulation_id')" >> "$GITHUB_OUTPUT"

      - name: Attach Digital Humans
        run: |
          curl -s --fail-with-body -X POST "$BLUEJAY_API/simulation/${{ steps.sim.outputs.id }}/add-digital-humans" \
            -H "X-API-Key: $BLUEJAY_API_KEY" \
            -H "Content-Type: application/json" \
            -d '{"digital_human_ids": [${{ vars.BLUEJAY_DIGITAL_HUMAN_IDS }}]}'

      - name: Run Bluejay Tests
        uses: bluejay-ai-dev/bluejay-github-actions@v1
        with:
          api_key: ${{ secrets.BLUEJAY_API_KEY }}
          simulation_id: ${{ steps.sim.outputs.id }}
          wait_for_results: 'true'
          min_score: '80'

      - name: Clean up
        if: always()
        run: |
          status=0
          if [ -n "${{ steps.sim.outputs.id }}" ]; then
            curl -s --fail-with-body -X DELETE "$BLUEJAY_API/simulation/${{ steps.sim.outputs.id }}" \
              -H "X-API-Key: $BLUEJAY_API_KEY" || status=1
          fi
          if [ -n "${{ steps.agent.outputs.id }}" ]; then
            curl -s --fail-with-body -X DELETE "$BLUEJAY_API/agent/${{ steps.agent.outputs.id }}" \
              -H "X-API-Key: $BLUEJAY_API_KEY" || status=1
          fi
          exit $status
```

> [!WARNING]
>
> The cleanup step deletes the agent and the simulation, and the run history with them. To keep results, remove the cleanup step or [test an existing agent](#test-an-existing-agent).

## Pin to a commit SHA

Some organizations require every third-party action to be pinned to a full-length commit SHA. If yours does, the workflow fails with *"all actions must be pinned to a full-length commit SHA"*. To find the commit for `v1`, run:

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
git ls-remote https://github.com/bluejay-ai-dev/bluejay-github-actions 'refs/tags/v1^{}'
```

Then put the SHA in place of `v1`:

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
uses: bluejay-ai-dev/bluejay-github-actions@c28260e8bd21f29e3841c712131a0ae979c7bdd7 # v1
```

> [!NOTE]
>
> A SHA pin does not follow updates to `v1`. To get fixes, run the command again and update the pin.

## Inputs

| Input | Required | Default | Description |
| - | - | - | - |
| `api_key` | Yes | None | Your Bluejay API key. |
| simulation\_id | Yes | None | ID of the simulation to run. |
| `prompt_id` | No | None | Agent version UUID to use for this run. |
| knowledge\_base\_id | No | None | Override Knowledge Base for this run. |
| digital\_human\_ids | No | None | Comma-separated list of Digital Human IDs. |
| phone\_number | No | None | Phone number to use for this run. |
| `sip_uri` | No | None | SIP URI to use for this run. |
| wait\_for\_results | No | `true` | If `true`, wait for the run to end and fail CI on the score. |
| `min_score` | No | `80` | Required pass rate (0–100). |
| poll\_interval\_seconds | No | `10` | Polling frequency in seconds. |
| timeout\_seconds | No | `1500` | Maximum wait in seconds (25 minutes). |

The step fails if the run does not complete, if the pass rate is below `min_score`, or if `timeout_seconds` passes first. To only queue the run without checking the score, set `wait_for_results` to `false`.

## Outputs

| Output | Description |
| - | - |
| `simulation-run-id` | The ID of the queued simulation run. |
| `final-status` | The final run status from the API, for example `completed`. |
| `score` | Pass rate of the run: `tests_passed / total_tests × 100`. |

## Triggers

To run the tests on a different event, change only the `on:` section. Any [GitHub Actions trigger](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows) works. For example, to run daily at 09:00 UTC:

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
on:
  schedule:
    - cron: '0 9 * * *'
```

## Related

* [Simulation API](/api-reference/endpoint/queue-simulation-run.md)
* [Simulation](/test/simulations/create.md)
* [Simulation Results](/test/simulations/create.md#read-the-results)
* [Bluejay as Code](/bluejay-as-code/overview.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.