025 - What Is a Context Window?

Paste a 200-page PDF into a chatbot and it answers questions about page 180 within seconds. That feels instant and free. Neither is quite true, and this episode explains what a context window actually is, what happens inside a model when it processes a huge document, and why a bigger window always comes with a cost. A context window is the total amount of text, measured in tokens, a model can hold at once across the prompt, any pasted document, and its own reply. Flagship models from OpenAI, Google, and Anthropic have all converged around a roughly one-million-token window in 2026, up from the 4,000- and 8,000-token limits those same companies shipped just a few years earlier. Inside that window, a transformer compares every token against every other token to decide what to attend to, so doubling the amount of text roughly quadruples the compute required, which is why OpenAI's own pricing sheet charges more once a request crosses into what it calls long-context territory. Size brings a second cost that has nothing to do with money: research on "lost in the middle" behavior shows models are measurably worse at using information buried in the center of a long context than information at the start or end, and newer testing on today's huge-context models found that gap gets worse, not better, as windows grow, with 10 to 25 percent accuracy loss on facts placed mid-document. The episode works through the actual math on a 200-page PDF: at roughly 500 words a page and OpenAI's own conversion rate of about 0.75 words per token, that document comes out to roughly 130,000 tokens and around 33 cents in input costs before the model writes a single word back. It also covers a concrete real-world response to that tradeoff: OpenAI's own coding tool, Codex, deliberately caps how much of GPT-5.6's advertised 1.05-million-token window it will actually use at once, capping it at 272,000 tokens specifically because of mid-document accuracy loss and runaway cost. The practical takeaway is simple: put what matters most at the start or end of a long prompt, not buried in the middle. Sources & References OpenAI, Key concepts (token-to-word conversion) — https://developers.openai.com/api/docs/concepts OpenAI, API pricing — https://developers.openai.com/api/docs/pricing Codex Knowledge Base, The Context Window Gap: Why Codex CLI Caps GPT-5.6's Million-Token Window at 272K — https://codex.danielvaughan.com/2026/07/20/context-window-gap-codex-cli-gpt56-advertised-vs-effective-budget-compaction-strategy/ hidekazu-konishi.com, LLM Context Window Growth Timeline — https://hidekazu-konishi.com/entry/llm_context_window_growth_timeline.html Voice narration is AI-generated.