Evals
Application evals — testing AI systems where the model is one component of a larger piece of software. Each entry is a repeatable technique for catching regressions, measuring accuracy, and finding the failures that create risk for the application and the people who depend on it. Not the foundational benchmarks that rank base models on leaderboards.
2 methods
Oracle
Benchmark evaluation
Score the model against a fixed, known-answer dataset so performance becomes a number you can track across versions and compare across models.
EstablishedEvalsBenchmarks
DifferentialDifferential testing
Run the same input across models or versions and treat divergence as a signal worth investigating.
EstablishedEvalsBenchmarks