Qlarify Labs
Evals
How we evaluate AI systems to find their limits. Each entry is a repeatable testing technique — the durable knowledge, independent of any one model or version.
Qlarify Labs
How we evaluate AI systems to find their limits. Each entry is a repeatable testing technique — the durable knowledge, independent of any one model or version.
4 methods
Judge the whole path an agent took, not just its final message — did it reach the goal, in how many steps, with which tools, and did it loop, stall, or wander.
Exercise the seams where the model meets the rest of the system — tool and function contracts, MCP handshakes, retrieval calls, API auth — to catch connectivity and contract failures end to end.
Specify invariants the output must always satisfy, then generate many inputs automatically and check the invariant on each.
Test the deterministic code around the model — prompt builders, output parsers, schema validators, tool wrappers — in isolation, with exact assertions, the way you'd test any software.