LLM-assisted input generation and fuzzing

How it works

Classic fuzzers mutate inputs at random and rely on speed and coverage feedback; they are strong at finding crashes but struggle to get past input validation into the logic behind it. A model brings knowledge of formats and meaning: it can write inputs that are syntactically valid but semantically odd — a date that exists in one calendar and not another, an address in an unusual script, a JSON document with duplicate keys, a file whose header contradicts its contents. Used together, the model seeds the fuzzer with a varied starting corpus, writes the harness or the format grammar, and proposes targeted inputs for areas the fuzzer's coverage shows it isn't reaching. Crashes, hangs, error responses and inconsistent outputs are the signals.

When to use it

Parsers, importers and file-handling code; public APIs and form-heavy applications; when a fuzzer is running but coverage has plateaued; and when setting up fuzzing for a format nobody has written a grammar for.

Limitations

Model-written inputs are slow and costly compared with a fuzzer, which tries orders of magnitude more inputs in the same time, so they complement fuzzing rather than replace it. The model gravitates towards well-known nasty inputs, which most mature code already handles. Without a good oracle beyond "didn't crash", the interesting failures — wrong answers to odd inputs — go unnoticed. Running generated inputs against shared environments can corrupt data or trigger real side effects.

What you still own

Where fuzzing is safe to run, what counts as a failure beyond a crash, and triaging what it finds.

Related evals

Cite this