Evals are the new unit tests
If a model change can reach production without a number turning red, you don’t have an AI system — you have a liability with a chat interface. How evaluation-driven delivery actually works.
No serious engineering organization ships code without tests. Yet organizations that would never merge an unreviewed pull request routinely swap the model behind a customer-facing AI system — a change that alters behavior on every single input — with nothing but a smoke test and optimism. The discipline gap is not conceptual; teams simply haven’t made evals as cheap and habitual as unit tests.
What an eval suite looks like in practice
An evaluation suite is a versioned set of real tasks with graded expected outcomes, run automatically on every change to the system — model, prompt, retrieval index, or tool. For a document-processing pipeline, that means a corpus of actual documents with verified extractions. For an Arabic customer assistant, it means real conversations across dialects with rubric-scored responses. The suite is the contract; everything else is implementation.
- Task-level metrics, not proxy benchmarks: your contracts, your tickets, your dialects — public leaderboards do not transfer.
- Graded severity: a formatting miss and a hallucinated clause are not the same failure, and the gate should know the difference.
- Regression thresholds agreed with the business, so “is this good enough to ship?” is a number, not a meeting.
- Continuous sampling from production feeding the suite, so the tests evolve with reality instead of fossilizing.
The organizational unlock
The technical benefit of evals is regression safety. The organizational benefit is bigger: evals convert arguments into measurements. Which model should we use? Run the suite. Is the cheaper model good enough? Run the suite. Did the vendor’s upgrade break anything? The suite already ran. Entire categories of steering-committee debate collapse into a dashboard — and vendor claims become checkable, which changes procurement conversations entirely.
Evals convert arguments into measurements. That is worth more than the regression safety.
Start smaller than feels serious: fifty well-chosen cases with honest grading beat five thousand scraped ones. Wire the suite into CI before the first production release, publish the trend line where the business can see it, and treat every incident as a new test case. Within a quarter, the suite becomes the single most consulted artifact in the program — the place where the truth lives.