There is a moment in the middle of a long conversation that everyone has had. The model forgets the rule you set at the start, repeats the thing you corrected a minute ago, and the answers get blurrier the further the conversation goes.
People usually describe it as "the model got dumber". What is actually happening is something else, and it has a name.
The model has no memory, it has text in front of it
A language model does not remember a conversation. Every time, it reads everything written up to that point as a single whole and produces the answer from that.
That pile of text is called the context, and it has an upper limit. The context window is exactly that limit.
As the chat gets longer the pile grows. When it reaches the limit the oldest parts start dropping off, and what has dropped is no longer in front of the model. If the instruction you gave at the start is in that dropped part, the model does not know it.
The surprising part starts before the window is full
Two separate things get mixed up here. One is the overflow just described. The other is a drop in quality that happens without any overflow at all.
A study by researchers at Stanford, UC Berkeley and Samaya AI measured the second one. Models do noticeably better when the information they need sits at the start or the end of the context, and worse when the same information sits in the middle. Overall accuracy also falls as the context gets longer, and that holds even for models designed for long contexts (Lost in the Middle, Liu et al.).
Paste in a fifteen-page document and ask about a detail in the middle of it, and you may get a worse answer than if you had given that detail on its own. The information is there. The model's attention does not land on it.
Why a bigger window is not the fix
The first fix that comes to mind is moving to a model with a bigger window. That solves only the first half of the problem.
It genuinely delays the overflow. It does nothing about information in the middle being missed, and it makes the problem worse by making it easier to put in more text.
There is a cost side too. Everything you put in the context is read again and charged again on every call. A document you carry along for no reason becomes a line you pay for over and over on every turn of that conversation. Latency climbs the same way.
The real work is choosing what to carry
This is why a job called "context engineering" has appeared in the last two years. It is less about finding the right sentence and more about putting the right information in front of the model at the right moment.
From where we sit, running agents, the picture is even sharper. Most of the text that enters the window during a run is not something the user typed. Tool definitions, the output of earlier steps, files that were read and intermediate notes take up most of the space.
Giving a sub-task a narrow briefing that is just enough for that job, rather than the whole history, turns out both cheaper and more accurate. Carrying more information is no substitute for carrying the right information.
Four habits that work on the user side
Start a new topic in a new chat. Old topics piling up in the same window do not help the new one, and they scatter attention.
Repeat the critical instruction near the end. If the rule you wrote at the start is buried in a long conversation, reminding the model of it alongside your request works.
Give the relevant section, not the whole document. If it turns out not to be enough you add more, but working the other way round is both more expensive and less accurate.
Summarise a long job and start over. The summary carries the conclusion reached so far and leaves the noise that built up in between behind.
For people building systems
The same principle holds in large setups. Everything that enters the window should have a reason, and things whose reason has expired should drop out.
Tool definitions are the most neglected item on that list. The definitions of tools the agent will never use also enter the context on every call, and they cost both space and money.
Carrying a long history as a summary rather than as-is works too. The price of a lost detail is usually lower than the price of the noise you carried.