Certain under-trained tokens cause models to emit nonsense, evade instructions, or behave erratically.
Published June 26, 2026
Reproducibility
Rare
Severity
Low
Confidence
Reviewer-confirmed
Details
Rare tokens that were present in the tokenizer but scarcely trained on (the 'SolidGoldMagikarp' family) trigger bizarre, off-distribution responses. Mostly low-severity reliability glitches, but they illuminate tokenizer/training mismatches.