Evaluating LLM systems
How we know an AI feature works, keep knowing it after every change, and hand you the numbers.
The principle
If we cannot measure it, we do not ship it. Every AI feature comes with an evaluation set, a harness that runs it, and a score you can read. The set is built from your real cases, grows while we work, and stays with you.
Building the set
- Collect real inputs, including the ugly ones: the badly scanned invoice, the ticket in three languages, the question with a typo. Fifty is a start; a few hundred is comfortable.
- Write expected outputs with the people who do the job today. For extraction this is the correct fields; for answers it is the facts that must be present and the ones that must not; for drafts it is a rubric.
- Tag the cases by type and difficulty so a score drop can be traced to a category, not just a number.
Running it
const report = await evals.run("support-drafts", {
model: "claude-opus-5-5",
cases: await evals.load("support-drafts@2026-10"),
graders: [factsPresent, noForbiddenClaims, toneRubric],
budget: { maxCostEur: 5 },
});
// report.score, report.byTag, report.regressionsThe harness runs in CI on every change to prompts, retrieval, tools or model version. A drop beyond an agreed threshold blocks the deploy, the same as a failing unit test.
What we measure
| Pattern | Metrics |
|---|---|
| Classification | Accuracy, per-class precision and recall, confusion on the costly mistakes |
| Extraction | Field-level exact match, schema validity, rate sent to review, review corrections over time |
| Retrieval and answer | Retrieval hit rate, citation correctness, answer faithfulness, "not in sources" when true |
| Drafting | Acceptance rate, edit distance, time saved per item, rubric scores |
| Tool use | Task completion, wrong-tool rate, steps and cost per task, approval-gate hits |
Plus, always: latency percentiles and cost per request.
In production
Evaluation does not stop at launch. A sample of real traffic is graded continuously, user feedback (thumbs, edits, escalations) flows back into the set, and a monthly report shows the trend. When a provider releases a new model version, the set tells us within an hour whether to switch.
What you receive
The cases, the graders, the harness and the history, in your repository, documented. Someone on your side can add a case in five minutes. That is the real deliverable; the model is replaceable.