Finding properties for property-based tests

How it works

Property-based testing replaces hand-picked examples with rules that should hold for any input, and lets a tool generate hundreds of inputs to try to break them. The hard part is thinking of the properties. Given a function, its documentation and its existing example tests, the model proposes candidates: encode-then-decode returns the original, sorting is idempotent, a total equals the sum of its parts, a result never exceeds a stated limit, an optimised implementation agrees with a simple one. It can also draft the generator for the input type and the test itself in the team's framework. Properties the tool falsifies are then examined: either a real bug, or a property that was never true.

When to use it

Parsers, serialisers, calculations, data transformations and anything with a clear mathematical shape; code with good example tests but suspected edge-case bugs; and teams that want to try property-based testing but don't know where to start.

Limitations

Many proposed properties are either trivially true or subtly false, and a false property produces failures that look like bugs until someone works out the property was wrong. The model favours textbook properties and is weaker at the domain-specific ones that catch real defects. Generators it writes can be too narrow to explore the space, which makes a passing run mean less than it appears to.

What you still own

Deciding which properties are genuinely required behaviour, and judging whether each failure is a bug in the code or in the property.

Related evals

Cite this