The judge
Judged suites use one model call per case, which covers every judged suite at once. The judge is told to be strict and fair and to return its verdict as JSON. It sees the question, the answer, the passages, the expected answer and your policy, and nothing else.Tool call arguments
A case passes when the app made a call with the expected name, and every expected argument matches:- Text is compared ignoring case and surrounding spaces.
- Objects match when every expected key matches. Extra keys from the app are allowed.
- Lists must have the same items in the same order.
Policy and tone
Write the policy in plain sentences. For example:Policy

