BASIL: Bayesian Assessment of Sycophancy in LLMs

Our summary

Proposes a Bayesian framework to separate genuine sycophancy from *rational* belief updating — a model changing its answer because a user supplied real evidence isn't sycophancy, and BASIL gives a formal way to tell the two apart by checking whether a model's revised belief is actually consistent with its own stated priors.

Why it matters

A real methodological risk in sycophancy probing is false positives — flagging any stance change as capitulation. BASIL gives probing a principled way to only count the irrational reversals.

Cited by these methods

Cite this