Skip to main content
A test set is a list of cases. Each case is one input to your app and what a good answer looks like. How many cases a test set can hold depends on your plan.

Fields

Column names are not case-sensitive, and spaces or hyphens count as underscores, so Expected Answer works.

CSV

cases.csv
Fields with commas or line breaks go in double quotes. A double quote inside a field is written twice.

JSONL

One JSON object per line:
cases.jsonl

From real questions

If you test a Context workspace, you can build a test set from the questions people already asked it. Evals takes the most recent ones as inputs; you add the expected answers.

Writing good cases

  • Use the words your users use, typos included.
  • Write expected as the facts that must be right, not as the exact wording.
  • Put a few out-of-scope questions in every set, marked should_refuse.
  • Keep must_contain to phrases that must appear verbatim, such as amounts and dates.