Testing & Findings

Findings

Documented limitations, weaknesses and failures of AI systems — evidence-first and linked to the method that found each one. Public entries are reviewed before publishing.

CategorySeverityMethodTagModelSearchClear

1 finding

MediumReasoningReviewer-confirmedRepro: Sometimes
Model behavior drifts between versions — a fixed task can regress
The same model name can perform very differently across dated snapshots: a task that passed on one release regresses on the next, with no announcement.
🔬 Drift & decay monitoring🔬 Differential testingReasoning failureEvalsDrift

Model behavior drifts between versions — a fixed task can regress