Testing & Findings

Testing methods

How to find the limits of AI systems. Each method is backed by the real findings it has surfaced — and ranked by how much it surfaces, weighted by severity. The durable knowledge is the technique, not the patched-away bug.

Most productive methods

  1. 1Differential testing7 findings · 7 versions
  2. 2Prompt-injection & jailbreak testing4 findings · 4 versions
  3. 3Boundary & edge-case testing7 findings · 6 versions

Ranked by a severity-weighted yield score. Why we measure this →

Clear

2 methods

Other1 finding · 4 versions

Distributional testing (KS test, Monte Carlo)

Sample the model many times and test the distribution of its outputs — not any single answer — for drift, miscalibration, or instability.

1 Medium
EmergingEvalsDrift
Differential2 findings · 1 version

Drift & decay monitoring

Re-run a fixed suite against each release and over time, watching for the quiet regressions and capability decay that a one-off evaluation can't see.

1 High1 Medium
EmergingDriftProduction