Tools

Declarative prompt and model evaluation from a config file, runnable locally and in CI. The closest thing to a regression suite for prompt changes: pin the cases, diff the results when the prompt or model moves.

Pytest-shaped evaluation for LLM output — assertions for relevance, faithfulness, bias and the rest. Fits teams that would rather express evals as unit tests than as a separate pipeline.

Retrieval-specific metrics — context precision and recall, faithfulness, answer relevance. Worth reaching for when a RAG system is wrong and you need to know whether retrieval or generation is at fault.

The UK AI Safety Institute's evaluation framework, built for the kind of evals it publishes itself — solvers, scorers, and tool-using agent tasks. Unusually legible about its own methodology.

EleutherAI's academic benchmark runner, and the tool behind many published benchmark numbers. Useful less for testing your own system than for reproducing a claim someone else made about a model.

A vulnerability scanner for LLMs — probes for prompt injection, jailbreaks, data leakage and toxicity, in the nmap sense of scanning rather than the bespoke sense of red teaming.

Microsoft's Python Risk Identification Toolkit — automates the repetitive half of red teaming (multi-turn attack orchestration and scoring) so the human effort goes into the attacks worth designing.

Scans LLM and tabular models for vulnerabilities and generates a test suite from what it finds. The generated suite is the interesting part — it turns a one-off scan into something that reruns.

Open-source tracing for LLM applications — nested spans over a chain or agent run, with evals and datasets attached. Answers "which step went wrong" for failures that only appear in production.

OpenTelemetry-based tracing and evaluation that runs locally in a notebook as readily as it does as a hosted service — a low-ceremony way to inspect agent trajectories while still developing them.

LangChain's hosted tracing, dataset and evaluation platform. Works with anything OpenTelemetry-instrumented, though it is most frictionless if you are already in the LangChain ecosystem.

NVIDIA's runtime rail layer — programmable constraints on what a conversation may do, defined in their Colang DSL. The rails themselves need testing, which is the part teams tend to skip.

Input and output validators with structure enforcement and automatic re-asking on failure. Overlaps with evaluation deliberately: the same checks can gate a response at runtime or score it in a test.

Open-source agentic end-to-end testing — an AI agent navigates your app and catches regressions on every PR, no test code required.

Playwright's built-in Planner, Generator, and Healer agents — the Healer auto-repairs broken UI locators but not data-level bugs.

AWS's agentic IDE/CLI; as a security-audit agent it's cheap and fast at flagging vulnerable files rather than writing tests.

Cognition's AI software engineer — strong at code-level analysis (e.g. finding untested endpoints), but blind to the DOM in UI testing.

Free, open-source AI pair programming in your terminal — generates test code with a feedback loop via a configurable test command.

Anthropic's agentic coding CLI — integrates into CI/CD via MCP for a closed feedback loop; a paid subscription tool.

Open-source, free alternative to Claude Code offering the same closed-loop agentic workflow with your choice of model.