Long context windows

Bigger context windows do not guarantee reliable use of everything in them. Models recall information best at the start and end of the input and worst in the middle, and performance degrades as the relevant span moves. Don't assume a large window means a fact buried mid-context will be used — the linked findings quantify the effect.