Needle-in-a-haystack (long-context retrieval)

How it works

By inserting a unique, easily-verified fact — a 'needle', such as a made-up sentence containing a specific number — at varying depths within contexts of increasing length, then asking the model to retrieve it, you map retrieval reliability as a function of two variables at once: how long the context is, and where in it the fact sits. Repeating this across a grid of lengths and positions produces a heatmap rather than a single pass/fail number, which is what reliably exposes the 'lost in the middle' pattern — strong recall near the start and end of a context, degraded recall in the middle — and separates a model's advertised context window from the length at which retrieval actually stays reliable.

When to use it

Evaluating a vendor's long-context claims before relying on them; sizing how much context a RAG or document-QA system can safely rely on the model to actually use, as opposed to merely accept as input.

Limitations

A synthetic, isolated needle is an easier retrieval task than reasoning that requires connecting or synthesizing multiple facts spread across a long context — passing the needle test is necessary but not sufficient evidence that real long-document tasks will succeed.

Cite this