Application evals — testing AI systems where the model is one component of a larger piece of software. Each entry is a repeatable technique for catching regressions, measuring accuracy, and finding the failures that create risk for the application and the people who depend on it. Not the foundational benchmarks that rank base models on leaderboards.
23 methods
Check that the system meets the transparency duties the law places on its provider and deployer — people told they are talking to an AI, generated content marked as such — in every channel, at the latest at the first interaction, and every time rather than most of the time.
Deliberately craft inputs designed to elicit failure — confusion, unsafe output, or broken constraints — to map the model's weak boundaries.
Score the model against a fixed, known-answer dataset so performance becomes a number you can track across versions and compare across models.
Measure outcome disparities across a population of demographically varied inputs, reporting fairness as statistics rather than anecdotes.
Test whether the reasons a system gives — its step-by-step reasoning, or the explanation it shows a user for a decision — actually determine its answer, or are a plausible story told afterwards.
Deliberately inject failures — tool timeouts, malformed tool responses, truncated context, adversarial inputs — to test whether the system degrades gracefully and recovers.
Hold a prompt fixed while swapping a protected attribute (name, gender, ethnicity) — the output should not change. When it does, you've measured bias.
Re-measure the accuracy, robustness and per-group performance a provider declares for its system — under the conditions the declaration names — and check the system actually meets what its documentation says.
Probe whether a deployed model can be cheaply queried to reconstruct its behaviour, training data, or a usable distilled copy — a confidentiality and IP attack surface.
Re-run a fixed suite against each release and over time, watching for the quiet regressions and capability decay that a one-off evaluation can't see.
For systems whose outputs feed back into their future inputs — retraining on their own decisions, retrieving their own generated content, ranking by engagement they produced — simulate the loop over many cycles and measure whether error or bias amplifies.
Test the human in the loop, not just the model — seed known errors into the system's output and measure whether the people overseeing it catch them, understand what it is doing, and can actually override or stop it.
Check whether a persuasive or engagement-driven system uses deceptive or manipulative techniques to steer people — and whether it treats users made vulnerable by age, disability or a social or economic situation differently in ways that exploit that vulnerability.
Treat the base model, fine-tuning data, embeddings, plugins, and every third-party package pulled into the pipeline as a software supply chain — provenance-checked and version-pinned, not merely capability-tested.
Before moving a working system to a newer model, replay what it did on the old one and check the new version keeps every behaviour you rely on — better on average is not the same as no worse anywhere.
Find out what people actually use the system for, beyond what it was built for, and test how it handles those uses — serve them well, decline them clearly, or redirect — rather than improvising.
Apply small, meaning-preserving changes to an input — typos, spacing, paraphrase, reordering — and check that the output stays stable. When it doesn't, you've measured brittleness.
Embed adversarial instructions in user input or retrieved/tool content to test whether the model follows attacker text over its system policy.
Probe the model and its outputs for PII, credentials, or proprietary data it was never meant to reveal — memorized training data, leaked context, or a secret pasted earlier in the same conversation.
Try to get the model to reveal its own system prompt or hidden instructions, then check whether that prompt held anything — a credential, an access-control rule, unfiled business logic — that mattered only because nobody expected it to leak.
Instrument the whole request — prompt, retrieval, model call, tool calls, tokens, latency, outcome — so real production behaviour becomes evidence you can query, sample, and score.
Look for the backdoors and bias a manipulated training, fine-tuning, or retrieval corpus can plant — a trigger phrase that flips behaviour, or a quality drift a routine corpus audit won't catch.
Probe the RAG pipeline's storage layer, not just its retrieval quality — can embeddings be inverted back to source text, can one tenant's query surface another tenant's documents, can a crafted document poison what gets embedded and returned.
