RESEARCH · CONTEXT & ATTENTION
The dumb
zone.
Give a model a long document and it reads the way you skim: the first lines land, the last lines land, the middle blurs. Put the one fact that decides the task in that middle, and the model looks straight past it. This failure is not random. It has a shape, and the shape is measurable.
Retrieval accuracy by position of the key fact · illustrative of the published effect
THE EFFECT
Recall is U-shaped, not flat.
A model does not weight a long context evenly. Accuracy at retrieving a specific fact is highest when that fact sits at the very start of the prompt, high again when it sits at the very end, and it sags in between.
Researchers named the dip lost in the middle. Move the answer from the edge of a long prompt to its center and retrieval accuracy can fall by double digits, with the same model, on the same question. The two edges get the attention. The middle pays for it.
The curve is not a quirk of one model. It appears across families and sizes, and it deepens as the context grows. More context is not automatically more understanding. Past a point, extra tokens dilute the ones that matter.
WHY IT HAPPENS
Attention has edges.
Recency is structural. These models generate each token from what came before, and the nearest tokens are the cheapest to attend to. The tail of the context sits right next to the answer the model is about to write, so it carries weight.
Primacy is learned. The opening of a prompt frames the task, and training text overwhelmingly states its point up front: abstracts, subject lines, topic sentences. Instructions and summaries cluster at the two edges of documents, almost never at the exact middle. The model absorbs that habit and expects the important thing to live near an edge.
The middle gets neither advantage. It is far from the generation point and out of distribution for where meaning usually sits. So it fades.
WHAT IT COSTS
When the middle is the business.
Enterprise inputs are middle-heavy by nature. The operative clause is on page 22 of a 40-page contract. The moment the client moves the deadline is ninety minutes into a two-hour transcript. The exception that changes the answer is one line inside a policy document. None of it is at an edge.
Naive retrieval makes it worse. Teams stuff the top fifty chunks into a prompt to be safe, and the one relevant chunk lands in the dead zone, outranked for attention by forty-nine near misses. A bigger context window does not fix this. It invites it.
WHAT TO DO
Engineer the position, not just the prompt.
The dumb zone is predictable, which means it is manageable. Four moves do most of the work:
Put decisive material at the edges.
Lead with the task and the facts it turns on. Restate the ask at the very end. Let the model's own biases carry your signal.
Retrieve less, rank harder.
A reranker that returns five strong passages beats one that returns fifty weak ones. Fewer tokens, better placed, higher accuracy.
Compress the middle.
Summarize supporting context so the ratio of signal to tokens stays high. A short, dense middle survives better than a long, loose one.
Keep the context tight.
A shorter, ordered prompt beats a longer, unordered one more often than teams expect. Length is a cost, not a feature.
HOW MILLIE MEASURES THIS
We do not trust a model's context window at its advertised size. Every long-context case in the benchmark plants a required fact at a controlled depth, then re-runs the same task with that fact moved from the edge to the middle. The score is the gap between the two.
The output is a per-model usable context length for documents like yours, not the spec-sheet number. A model that only works when the answer is on page one is not a model you can hand a contract. See what we measure.