Understanding the Context Window: Why Size Is Not Everything

1. What a context window actually is

A context window is the maximum number of tokens a language model like Claude can process and relate to one another within a single request. That includes the system prompt, the entire conversation history, all tool outputs, and every file's content embedded in the request. The key distinction is that a context window is not memory: it is not persistent storage across sessions but a working buffer per request, rebuilt from scratch from whatever data is sent along with each new call. Nothing outside this window influences the current answer, no matter how relevant it would be.

The difference between tokens and words matters in practice. A token corresponds to roughly four characters of English text, while other languages with different morphology and compound words can tokenize somewhat differently. Source code tokenizes differently again from prose: indentation, brackets, special characters, and long variable names produce more tokens per line than natural language. Anyone who thinks about context size purely in characters or lines routinely underestimates how quickly a window fills up with larger files or long log output.

2. Context window sizes at a glance

Current Claude models offer a standard context window of 200,000 tokens, and for select models and use cases an extended window of up to one million tokens is available through a beta feature. For comparison, 200,000 tokens is roughly equivalent to 500 pages of text or several thousand lines of source code, depending on the language and formatting. That sounds generous, but it fills up surprisingly fast once several large files, a long conversation history, and tool outputs such as test logs or build output add up.

It is important to separate input context from the maximum output length. The context window limits how much the assistant can read and factor into its answer, while the output limit independently caps how long a single response can be. The two do not share the same budget, but together they determine how much structured code a model can meaningfully deliver in a single request. Anyone expecting a huge refactor to come back in one single response is more likely to hit the output limit than the context window itself.

#!/usr/bin/env bash
# Claude Code: current token usage and session cost for the active session
claude
> /cost
# Token usage:  118,432 / 200,000 (59%)
# Cache reads:   84,120 tokens (cached, cheap)
# Cache writes:  12,300 tokens
# Session cost:  $0.94

# Compact the conversation when it grows large but the topic is still relevant
> /compact
# Summarizing conversation history into a condensed context...
# Context reduced from 118,432 to 21,050 tokens

3. Why size does not equal quality

A larger context window does not mean the model understands or weighs every token equally well. Internally, the attention mechanism spreads attention across all tokens in the window, and the more tokens are present, the more each individual token competes for relevance. Research on this effect, often referred to as "lost in the middle," shows that models retrieve information from the beginning and end of a long context more reliably than information buried somewhere in the middle.

In practice, this means that copying an entire codebase with hundreds of files into a request and then asking about a single bug in a specific function increases the risk that exactly that function gets lost in the attention distribution. A smaller, deliberately curated context containing only the actually relevant files often produces sharper answers than a huge but unstructured one, even when the token budget would theoretically be sufficient. Size does not substitute for relevance.

4. How tokenization fills the context window

Claude uses a byte-pair-encoding-based tokenizer that breaks text into subwords or individual characters, depending on how frequently a given character sequence appeared in the training corpus. Common English words often become a single token, whereas rare technical terms, camelCase variable names, or generated IDs split into several tokens. A path like node_modules/@vendor/package/dist/index.min.js can burn through a dozen tokens by itself before any actual application code is even read.

That is why it pays to roughly estimate how many tokens a file or directory actually requires before a large request, rather than relying on file size in kilobytes. The Anthropic API offers a dedicated counting endpoint that returns the exact token count of a planned request before any cost is incurred. Anyone working regularly with large repositories should consistently exclude generated directories such as vendor, node_modules, or build artifacts from the context, since they consume tokens without adding meaningful content.

# Estimate token usage before sending a large request
import anthropic

client = anthropic.Anthropic()

with open("app/code/Mironsoft/SeoSuite/Model/MetaGenerator.php") as f:
    file_content = f.read()

response = client.messages.count_tokens(
    model="claude-sonnet-4-5",
    system="You are a senior PHP/Magento developer.",
    messages=[
        {"role": "user", "content": f"Review this file:\n\n{file_content}"}
    ],
)

print(f"Estimated input tokens: {response.input_tokens}")
# Estimated input tokens: 3842
# Run this check before dumping an entire directory into a prompt

5. Prompt caching: cost and latency for reused context

Prompt caching addresses a practical problem: sending the same large system context again in multiple successive requests, for example a CLAUDE.md file or several reference files, costs money and processing time for the full context every single time. Blocks marked with cache_control are cached server-side, so subsequent requests with an identical prefix do not need to reprocess that portion in full. This meaningfully reduces both latency and cost, especially in iterative coding sessions with a stable project context.

It is worth stressing that prompt caching is not a replacement for a well-considered context window, only an optimization on top of it. The cached context still has to fit within the context window, and a bloated, unstructured context remains an attention problem even with caching enabled. Caching pays off mainly for stable, rarely changing content placed at the beginning of a prompt, while frequently changing user requests belong at the end, since any change ahead of a cache block invalidates its reusability.

{
  "model": "claude-sonnet-4-5",
  "system": [
    {
      "type": "text",
      "text": "Project context: Magento 2.4.8, PHP 8.4, Hyva Theme, PHPStan Level 5. [... full CLAUDE.md content, several thousand tokens ...]",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    {
      "role": "user",
      "content": "Fix the null pointer issue in MetaGenerator::generate()"
    }
  ]
}

6. Context management in Claude Code

Claude Code addresses the problem of an overloaded context structurally by giving the model targeted tools instead of a blanket full-text dump. Rather than copying an entire codebase into the prompt, the agent uses Grep and Glob to locate relevant files by pattern and search term, then reads only the actually needed slices using targeted offsets and line limits. This grep-first approach keeps the context small and focused, even in repositories with tens of thousands of files.

A CLAUDE.md file at the project root complements this approach by providing stable project knowledge, such as coding standards, directory structure, and recurring commands, condensed once instead of re-explaining it every session. This drastically reduces context overhead compared to inserting the same explanatory text or entire reference files into every single request. The biggest anti-pattern in practice remains common regardless: dumping an entire directory into the prompt via cat or copy-paste, in the hope that "more context equals a better answer."

#!/usr/bin/env bash
# Grep-first: find the relevant file before reading anything into context
rg -l "MetaGenerator" app/code/Mironsoft/SeoSuite --type php

# Read only the matching lines plus a bit of surrounding context
rg -n -A 5 -B 5 "function generate" \
  app/code/Mironsoft/SeoSuite/Model/MetaGenerator.php

# Anti-pattern to avoid: dumping the whole module into the prompt
# cat app/code/Mironsoft/SeoSuite/**/*.php > /tmp/everything.txt

7. Sub-agents and context isolation

For complex tasks such as large refactors, code reviews spanning multiple modules, or research over an unfamiliar codebase, context isolation via sub-agents is a useful pattern. A sub-agent receives its own lean context window scoped only to its specific subtask, works within it independently of the main context, and returns a condensed result at the end instead of writing the entire research trail into the parent context. This keeps the main context lean and focused across a long session.

This pattern is particularly valuable when a task requires many intermediate steps with high token consumption, such as searching dozens of files for a pattern, whose result at the end only needs a few lines of summary. Without a sub-agent, all of those intermediate steps would fill the main context and dilute attention away from the actual task. With a sub-agent, only the relevant final result stays visible in the main context, while the expensive search happens in a separate, disposable context.

8. Session hygiene: when to compact, when to restart

Even within the available token budget, long sessions accumulate clutter: discarded approaches, stale error messages, long-fixed bugs, and intermediate states that no longer matter for the current task but still consume attention. Claude Code offers the /compact command for this, which condenses the prior history into a compact summary while keeping only the information still relevant to further work. This differs from /clear, which fully resets the entire context and is intended for a genuine change of topic.

The practical rule of thumb: if a new task has nothing to do with the previous one, for example switching from a checkout bugfix to a new backend feature, a /clear is usually worth more than a /compact, even if plenty of token budget theoretically remains. A clean restart with deliberately loaded context produces more reliable results than a long session that is still technically within budget but mentally carries several unrelated topics along with it.

// Simple heuristic to flag files that would consume a large token share
// before including them in a prompt (rough estimate: ~4 chars per token)
const fs = require('fs');

function estimateTokens(filePath) {
  const content = fs.readFileSync(filePath, 'utf8');
  return Math.ceil(content.length / 4);
}

function shouldWarnBeforeIncluding(filePath, budgetTokens = 200000) {
  const estimated = estimateTokens(filePath);
  const threshold = budgetTokens * 0.05; // more than 5% of the window
  if (estimated > threshold) {
    console.warn(
      `${filePath}: ~${estimated} tokens, consider reading a slice instead`
    );
  }
  return estimated;
}

shouldWarnBeforeIncluding('var/log/exception.log');

9. Anti-patterns versus proven strategies

The following comparisons summarize which habits unnecessarily burden the context window and which alternatives produce more reliable results in practice.

None of these patterns is purely a question of raw token count. A context window with plenty of free budget can still produce poor answers if the content is unstructured and irrelevant, while a tighter but deliberately curated context often produces the more precise result. Choosing the right strategy depends on the task, not on the mere availability of tokens.

10. Summary

A context window is fundamentally a token budget per request, not a substitute for memory and no guarantee of quality by size alone. 200,000 to one million tokens sounds generous, but it fills up quickly with source code, logs, and long conversation histories, and even within budget, reliability drops when relevant information gets lost in an overloaded context. Tokenization treats code differently from prose, and prompt caching reduces cost and latency for reused context, but it does not replace deliberate curation.

In Claude Code, this becomes practical: grep-first search instead of full-text dumps, a CLAUDE.md for stable project knowledge, sub-agents for context isolation on complex subtasks, and /compact or /clear for session hygiene all keep the context focused. Anyone who consistently uses these tools gets more reliable answers with lower token consumption, regardless of how large the theoretical window of the model in use actually is.

11. FAQ: Understanding the Context Window