Sycophancy Eval — Inspect Evals

Our summary

An open-source, automated 'are you sure?' probe built on AISI's Inspect framework: it challenges a model after a correct answer and checks whether it holds its position, apologizes for a mistake it didn't make, or reverses course.

Why it matters

A ready-made, maintained open-source harness for running this probe at scale, instead of hand-writing bespoke challenge prompts each time.

Cited by these methods

Cite this