Predictive test selection

How it works

The baseline to beat first is plain dependency-based selection: run the tests that depend on the changed code. Predictive selection goes further. A model is trained on history: for each past change it sees the files touched, how far the dependencies are from each test, each test's own failure record, and whether the test failed. For a new change, it ranks tests by how likely they are to fail, and CI runs down the list until it reaches a budget. This is classic machine learning rather than an LLM, and large engineering organisations have published on running it in production. The full suite still runs on a schedule to catch what the selection skipped, and those misses feed back into training.

When to use it

Suites too slow to run on every change, large monorepos, and teams where CI cost or queue time is the bottleneck. It needs plenty of history containing real failures.

Limitations

It learns from the past, so new kinds of change, new tests and rare failures get underweighted. Flaky tests corrupt the training labels. A skipped test that would have failed stays invisible until the next full run. Small teams may not have enough history to train on, and proven at very large scale doesn't mean worth it at small scale.

What you still own

How often the full suite runs, which tests are never skipped, and keeping an eye on the miss rate.

Cite this