Qlarify Labs
Evals
How we evaluate AI systems to find their limits. Each entry is a repeatable testing technique — the durable knowledge, independent of any one model or version.
Qlarify Labs
How we evaluate AI systems to find their limits. Each entry is a repeatable testing technique — the durable knowledge, independent of any one model or version.
7 methods
Deliberately inject failures — tool timeouts, malformed tool responses, truncated context, adversarial inputs — to test whether the system degrades gracefully and recovers.
Probe whether a deployed model can be cheaply queried to reconstruct its behaviour, training data, or a usable distilled copy — a confidentiality and IP attack surface.
Treat the safety filters around the model as their own system under test — measuring bypass rate on unsafe inputs, false-positive cost on benign ones, and what happens when the guardrail itself fails.
Deliberately steer the model toward fabrication — asking about non-existent entities or beyond its knowledge — to map where it invents instead of declining.
Apply small, meaning-preserving changes to an input — typos, spacing, paraphrase, reordering — and check that the output stays stable. When it doesn't, you've measured brittleness.
Deliberately probe whether a model abandons a correct answer or endorses a false one when the user pushes back, to catch approval-seeking behavior that plain accuracy evals miss.
Walk inputs across a decision boundary — refusal, classification, confidence cutoff — to find exactly where the model's behaviour flips, and whether it flips in the right place.