Context Windows Explained

Overview

The context window is the single most important constraint governing how AI coding agents work. It determines what the agent can see, how much code it can reason about at once, and ultimately how effective it is at understanding your codebase.

Think of the context window as the agent's working memory -- a fixed-size desk where everything the agent needs to think about must fit at the same time. When the desk is full, something has to fall off. Understanding this constraint will change how you structure your code and documentation.

What Is a Context Window?

A context window is the total amount of text (measured in tokens) that a large language model can process in a single conversation. Everything the agent needs to reason about must fit within this window:

• The system prompt (agent instructions, tool definitions)
• The conversation history (every message, tool call, and tool result)
• The current input (the latest tool output or user message)
• The output space (room for the model to generate its response)

The context window (e.g., 200K tokens) is divided into layers: System Prompt (2,000-5,000 tokens), Project Docs like CLAUDE.md (500-3,000 tokens), Conversation History (grows with each turn), Recent Tool Results (file contents, search results), and Output Space (4,000-8,000 tokens reserved).

Tokens: The Unit of Measurement

Models do not count characters or words -- they count tokens. A token is a chunk of text, roughly corresponding to:

• 1 token is approximately 4 characters or 0.75 words in English
• 1 line of code is approximately 3-5 tokens
• A typical source file (200 lines) is approximately 600-1,000 tokens
• A large source file (1,000 lines) is approximately 3,000-5,000 tokens

Here is a concrete example of how tokenization works:

A model with a 200,000 token context window sounds enormous -- and it is. But consider a real project:

This is why 200,000 tokens still is not enough for large codebases. The context fills up not because of the code you want to change, but because of all the code the agent needs to understand to make that change.

How Agents Fill the Context Window

Let us trace how the context window fills during a real task:

How context fills across three phases: Turns 1-3 (Exploration) add ~2,600 tokens reading the README, searching, and reading the target file. Turns 4-7 (Understanding) add ~3,500 tokens reading related, test, types, and config files. Turns 8-10 (Acting) add ~3,400 tokens for editing, running tests, and fixing failures.

After just 10 turns, the agent has consumed roughly 9,500 tokens of conversation history. That is manageable. But consider what happens during a complex feature implementation with 40+ turns, large files, and verbose test outputs -- the context can fill rapidly.

What Happens When Context Overflows

When the conversation approaches the context limit, the agent framework must make a choice. Different agents handle this differently, but the common strategies are:

• Truncation: Older messages are dropped from the beginning of the conversation. The agent loses its memory of early exploration and decisions.
• Summarization: Earlier parts of the conversation are compressed into a summary. Details are lost, but the gist is preserved.
• Sliding window: Only the most recent N messages are kept. Everything older is discarded.

All of these strategies have the same consequence: the agent forgets.

Before truncation: Turns 1-4 contain detailed exploration of architecture and modules, followed by implementation work through Turn 16. After truncation: Turns 1-4 are lost, replaced by a brief summary ('Explored architecture and module structure'), with only Turns 5-16 preserved in full.

When an agent forgets its early observations, it might:

• Re-read files it already read, wasting more tokens
• Contradict earlier decisions, introducing inconsistency
• Lose track of the overall plan, solving the wrong problem
• Forget constraints it discovered, like "don't modify the public API"

The "Lost in the Middle" Effect

Even before the context overflows, there is another problem. Research has shown that LLMs pay uneven attention to different parts of the context window. Information at the beginning (system prompt, early instructions) and the end (recent messages) gets the most attention. Information in the middle receives less (see Liu et al., "Lost in the Middle: How Language Models Use Long Contexts," 2023).

This is called the lost in the middle effect, and it has practical consequences:

• A file read 20 turns ago is partially "forgotten" even if it is technically still in the context
• Critical instructions buried in the middle of a long CLAUDE.md may be overlooked
• The agent may ask for information it already has, because that information is in a low-attention zone

Practical tip: Keep your CLAUDE.md concise and front-load the most important instructions. The agent pays the most attention to the beginning and end of its context.

How Code Structure Affects Context Efficiency

This is where agent-readiness becomes practical. The way you structure your code directly impacts how efficiently agents use their context window.

A 2,000-line file forces the agent to load 6,000-10,000 tokens just to read one file. If the agent only needs to understand one function in that file, 90% of those tokens are wasted.

A 200-line file with a single responsibility costs only 600-1,000 tokens. The agent loads exactly what it needs.

Comparison of two code structures: A monolithic utils.py (2,000 lines, ~7,000 tokens) forces the agent to read the entire file to find one function. A modular structure with validation.py (150 lines, ~500 tokens), formatting.py (120 lines, ~400 tokens), and parsing.py (180 lines, ~600 tokens) lets the agent read only the file it needs.

The same principle applies to:

• Function length: A 200-line function is harder for agents than five 40-line functions
• Module organization: Flat file structures force agents to read more files to find things
• Import structure: Clear, explicit imports help agents trace dependencies without reading everything
• Documentation placement: Inline docstrings mean the agent gets context for free when it reads the code

Strategies for Staying Within Budget

Given the context window constraint, here are concrete strategies for making agent interactions more efficient:

• Keep files focused: One module, one responsibility. Aim for under 300 lines per file.
• Write descriptive names: Functions called process_data force the agent to read the body. Functions called validate_email_format tell the agent what they do from the name alone.
• Front-load documentation: Put the most important information at the top of CLAUDE.md and README files.
• Use explicit exports: An __init__.py or index.ts that re-exports public APIs helps agents find what they need without reading every file.
• Structure for searchability: Consistent naming patterns let agents find things with search instead of reading entire directories.

Key insight: Every token you save through better code organization is a token the agent can spend on understanding, reasoning, and producing better edits.

Summary

The context window is the finite working memory every AI agent operates within. Understanding it reveals why some codebases are easier for agents to work with:

• Tokens are the currency: Every file read, every search result, every tool output costs tokens
• Context overflow causes forgetting: When the window fills, the agent loses its earliest observations
• The middle gets less attention: Front-load important information in docs and code
• Code structure is context efficiency: Smaller, focused files mean the agent loads less to understand more
• Budget your tokens: Treat context like a scarce resource and design your codebase accordingly

In the next lesson, we will examine the specific tools agents use to interact with your code -- and how understanding tool mechanics will help you write more agent-friendly code.