XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models

Our summary

A test suite of clearly-safe prompts designed to surface exaggerated safety behaviour: models refuse benign requests that merely resemble unsafe ones or mention sensitive words.

Why it matters

Over-refusal is a measurable usability failure of safety tuning, and XSTest gives a concrete way to catch it.

Cited by these methods

Related findings (1)

• Over-refusal of benign requestsMedium

  Safety tuning causes refusal of harmless requests that merely resemble sensitive ones.

Cite this