Lost in the middle: why a 100k-token context window is not 100k tokens of attention

An LLM does not read a long context evenly. Bury a single fact — a needle — inside a long stack of documents and ask the model to retrieve it, and how reliably it succeeds depends heavily on where the fact sits. Facts at the very start (primacy) or the very end (recency) are recalled reliably; a fact in the middle is far more likely to be missed. Plot accuracy against needle position and you get a U-shaped curve — high at both edges, sagging in the middle. That's the finding of Liu et al.,…