A window of a million tokens begs you to load the whole project into it. Most of the time you should not, and here is why.
What a large context does
Three effects show up in practice.
Cost grows linearly. Each request pays for the entire context again. In an agentic run with dozens of calls that bill multiplies.
Attention gets diluted. The model finds a particular detail less reliably when a hundred thousand lines of unrelated code surround it. A smaller but precise context often gives the better answer.
Latency goes up. Reading a large input takes time as well.
When long context is justified
- the task needs links between distant parts, and you cannot say beforehand which ones
- the document is whole and cutting it makes no sense: a contract, a specification, the log of one long session
- a one-off analysis where quality matters more than price
When to cut it down
- the question is narrow and you know where the answer lives
- processing runs in batches, and the same large chunk would travel with every request
- the task repeats often while the content barely changes
In the last two cases it pays to cache the stable part, or to retrieve the relevant fragment and send only that.
A simple rule
Picture solving the task by hand. Would you reread the whole project, or open three files?
If it is three, give the model three. The other hundred will not make it smarter.
What it costs
Multiply the context size by the number of calls and by the input rate. In an agentic task of twenty steps, the gap between "the whole repository" and "only the files that matter" often reaches an order of magnitude.
The calculator on the site lets you run it on your own numbers.
What to do in practice
- Start with a small context and add more when the model asks for it.
- Put the stable part in a cacheable prefix and keep the variable part at the end.
- For repeating tasks, keep an index and insert only the relevant fragments.
Short version
A large window is an option, not an instruction. Send what the task needs, not everything that fits.