Measuring and Improving Consistency in Pretrained Language Models

Our summary

Using paraphrased cloze queries (ParaRel), shows that pretrained models give inconsistent answers to logically equivalent questions — a poorly structured knowledge representation rather than a stable one.

Why it matters

Empirical grounding for logic and consistency testing — asserting the invariants a sound reasoner must satisfy and watching them break across related questions.

Cited by these methods

Cite this