Open Evals at jabrod.com/app/evals.
1
Add a target
Tell Evals how to call your app: its URL, any headers, a JSON body with Answer path:
{{input}} where the question goes, and where the answer is in the response. See Targets.Body
choices.0.message.content2
Upload a test set
A CSV or JSONL file, one case per row. Only
input is required. See Test sets.cases.csv
3
Pick suites and run
Choose the suites that apply. A run calls your app once per case and scores the answers. One run per account at a time.
4
Read the report
Pass rate by suite, then every case with its answer, the passages it used, and a reason for each verdict. See Reports.

