Skip to main content
The CI hook is part of the Evals Pro plan.
Two HTTP endpoints let a CI job start a run and read its result. Both take a personal API key from jabrod.com/app/keys, stored as a CI secret:

Start a run

It answers 202 with {run_id, status: "queued", url}. url opens the run’s report in the Evals app.

Read the result

Response
status is queued, running, done or failed. previous_pass_rate is the last finished run of the same test set against the same target. A failure with regression: true passed that suite last time.

Example: GitHub Actions

.github/workflows/evals.yml
The last line fails the job when the run failed or any case regressed. Change it to compare pass_rate with a threshold if you prefer.

Errors