Glitch-token & unicode fuzzing

How it works

Certain tokens exist in a model's vocabulary but were scarcely represented during training — an artifact of how the tokenizer and training corpus were built — and feeding them to the model can trigger bizarre, off-distribution completions: nonsense output, instructions being ignored, or the model behaving as if it lost its place. Separately, homoglyphs (visually near-identical unicode characters from different scripts), zero-width characters, and unusual encodings can be used to write text that reads normally to a person but doesn't match the literal string patterns a keyword filter is checking for. Fuzzing systematically sweeps rare token IDs and unusual unicode sequences through the input, flagging both classes at once: reliability glitches from under-trained tokens, and a genuine safety-bypass surface from encoding obfuscation.

When to use it

Robustness hardening against unusual or malformed input; evaluating whether safety filters key on literal text patterns that encoding tricks can dodge; designing input-sanitization and normalization before it reaches the model.

Limitations

Findings are often version- and tokenizer-specific, so a glitch token or bypass sequence found against one model or release can be ephemeral — retraining or a tokenizer change can silently fix or move it, and testing needs to be re-run after either.

Cite this