Context Window Errors Across All AI Providers (2026) — Complete Directory — AI Error Hub

Category hub · Cross-provider · Context window

Context Window Errors Across All AI Providers — 2026 Guide

Context window errors get worse with model tier — moving from GPT-4 (128K) to GPT-5.6 (400K) to Gemini 3.5 Pro (1M) doesn't eliminate the problem, it just shifts where it fires. Every model has a ceiling; every workload hits it eventually. This is the cross-provider directory: every context window failure pattern grouped by provider and layer.

Curated by Zain Khan

Editor, AI Error Hub

Updated October 7, 2026

Context window errors get worse with model tier — moving from GPT-4 (128K) to GPT-5.6 (400K) to Gemini 3.5 Pro (1M) doesn't eliminate the problem, it just shifts where it fires. Every model has a ceiling; every workload hits it eventually. Long conversations accumulate. System prompts and tool definitions count. Structured output schemas count. Extended thinking counts. This page is the cross-provider directory: every context window failure pattern we've documented across every provider, IDE tool, and gateway, grouped so you can find your specific error in seconds.

For architectural patterns to prevent context errors in the first place, see the context window chapter of the Cost Optimization Playbook. For rate limit patterns often triggered by large-context requests, see the Rate Limits category hub. For related failure modes, see the Authentication category hub.

Anthropic Claude OpenAI Google Gemini AWS Bedrock Azure OpenAI IDE Tools Gateway Layer Framework Layer

2026 context windows at a glance

Context windows have grown 8x since 2023. Here's the 2026 landscape at a glance.

Model	Max input	Max output	Cache mechanism	Streaming
Claude Sonnet 4.7	200K tokens	16K tokens	cache_control (5 breakpoints)	Yes
Claude Sonnet 5	1M tokens	16K tokens	cache_control (5 breakpoints)	Yes
Claude Opus 4.7	200K tokens	16K tokens	cache_control (5 breakpoints)	Yes
Claude Haiku 4.5	200K tokens	16K tokens	cache_control (5 breakpoints)	Yes
GPT-5.6 Terra	400K tokens	16K tokens	Automatic (≥1024 tokens)	Yes
GPT-5.6 Sol	400K tokens	16K tokens	Automatic (≥1024 tokens)	Yes
GPT-5.6 Luna	128K tokens	16K tokens	Automatic (≥1024 tokens)	Yes
Gemini 3.5 Pro	1M tokens	65K tokens	Explicit context caching	Yes
Gemini 3.5 Flash	1M tokens	8K tokens	Explicit context caching	Yes
Bedrock (varies)	Model-specific	Model-specific	Model-specific	Yes
Azure OpenAI (varies)	Deployment-specific	Deployment-specific	Automatic (≥1024 tokens)	Yes

Anthropic Claude

Claude has two context windows in production in 2026 — 200K (Sonnet 4.7, Opus 4.7, Haiku 4.5) and 1M (Sonnet 5). Output is capped at 16K on all variants. The most common Claude context errors involve input overflow, extended thinking budget conflicts, and prompt caching breakpoint limits.

→ Claude API Pillar Guide

OpenAI

GPT-5.6 Terra and Sol offer 400K context; Luna is 128K. Reasoning tokens (from reasoning_effort) count against output — they're invisible in the response but consume budget. Structured output schemas beyond a certain size can push you over the context limit indirectly.

→ OpenAI API Pillar Guide

Google Gemini

Gemini 3.5 Pro's 1M-token context is real — needle-in-a-haystack testing shows reliable recall at that scale. Gemini 3.5 Flash also carries 1M input but only 8K output. File API uploads count toward context via automatic tokenization; large PDFs and videos consume proportionally.

→ Gemini API Pillar Guide

AWS Bedrock

Bedrock context varies per model — Claude models inherit Anthropic's limits, Titan and Llama have their own. Cross-region inference profiles don't change context size but do affect payload routing. Streaming responses have separate payload size limits that can fire even within context.

→ Amazon Bedrock Pillar Guide

Azure OpenAI

Azure OpenAI context follows the underlying OpenAI model but with per-deployment configuration. Global deployments and regional deployments can have different context handling. PTU deployments enforce context limits identically to Standard.

→ Azure OpenAI Pillar Guide

Context management at other layers of the stack

Context management isn't only a provider-layer concern. IDE tools compact aggressively, gateways normalize context parameters, frameworks accumulate state — each layer has its own failure modes.

IDE Tools

IDE tools (Cursor, Claude Code, GitHub Copilot) manage context differently than raw API calls — they use aggressive compaction, workspace indexing, and file-scoped context. Context errors in IDE tools manifest as quality degradation more often than hard failures. The tool 'keeps working' but with less context than you expect.

→ IDE Tool Comparison Post

Gateway Layer

Gateways add their own context handling on top of the underlying provider. OpenRouter normalizes max_tokens across providers (some providers use max_tokens, others max_completion_tokens). LiteLLM's context_window_fallbacks route to larger-context models on overflow. Portkey configs can inline system prompts that push you over limits.

→ AI Gateway Comparison Post

Framework Layer

Frameworks generate more tokens per user turn than direct API calls do. LangChain conversation memory accumulates. LangGraph state can grow unboundedly if not pruned. LlamaIndex node reranking window controls how many chunks feed the LLM. Framework context errors are almost always state-management errors.

→ LangChain Pillar Guide

Frequently asked questions

Why do I still hit context errors on 1M-token models?

Three reasons. (1) Advertised context and reliable context are different — most 1M models degrade in recall past 60-70% utilization. (2) System prompts, tool definitions, and structured output schemas all count toward the ceiling. (3) Long conversations accumulate — 20 turns at 20K tokens each = 400K just from history. The ceiling is real even when it seems infinite.

What actually counts toward my context window?

Everything the model sees: system prompt, tool definitions, message history (all turns), current user message, images (~85 tokens per image plus per-tile), files loaded via File API, and structured output schemas. Some providers also count extended thinking / reasoning tokens toward context (not just output). Instrument the token count per request to see what your workload actually consumes.

How do I reduce input tokens without losing information?

Prompt caching is the highest-ROI single fix — repeated context is billed at 10% of standard rate on Claude, 25% on Gemini, 50% on OpenAI. Beyond caching: summarize old conversation turns, use RAG instead of stuffing entire documents, and trim tool definitions to what's actually used. See our Cost Optimization Playbook Chapter 3 for the specifics.

What's the difference between max_tokens and context window?

Context window = total tokens the model can process at once (input + output combined for most providers). max_tokens = the cap on output tokens specifically. You can hit max_tokens (output cap) without hitting context window (total cap). Both cause errors but with different fixes.

Why does my Claude request fail with 'input too long' but Sonnet 5 has 1M context?

Check that you're actually using Sonnet 5 (model ID: claude-sonnet-5-5-20260228 or similar). Sonnet 4.7 keeps the 200K limit. Also, extended thinking budget_tokens count toward the input side of context — a 200K prompt plus 32K thinking budget doesn't fit in Sonnet 4.7's 200K window.

How does Gemini's 1M context compare to Claude's 1M context in practice?

Both are real, both are usable, and both degrade before the ceiling. Gemini 3.5 Pro has the longer track record at 1M scale and slightly better needle-in-a-haystack scores. Claude Sonnet 5 matches or beats Gemini on multi-hop reasoning across long contexts. For typical RAG workloads under 300K tokens, they're indistinguishable in practice.

Should I upgrade to a larger-context model to fix context errors?

Usually no — usually the fix is to reduce input, not to raise the ceiling. A 1M-token workload costs 5-10x more per request than a well-designed 100K-token workload doing the same job. Instrument input tokens per request first. If you're at 40%+ of your current model's ceiling on average, then upgrade context and cost together.

How do I handle context errors in agent workloads?

Delegate to subagents with independent context windows. Claude Code's subagent pattern, LangGraph subgraphs, and LlamaIndex nested workflows all support this. Main agent dispatches heavy work to subagent, subagent uses its own context budget, main agent receives only the summary. This is the standard pattern for workloads that would otherwise blow the main context window.

What does 'auto-compact thrashing' mean?

IDE tools like Cursor and Claude Code automatically compact conversation context when it approaches the limit. Thrashing happens when compaction frees space that gets immediately refilled — e.g., reading a large file, compaction, reading the same large file again, compaction again. The tool 'keeps working' but with significantly degraded context awareness. See our Cursor and Claude Code error pages for the specific patterns.

How do I test whether my model handles long context well?

Run a needle-in-a-haystack test on your actual workload. Insert a specific fact at various depths in your typical input (25%, 50%, 75% into the context) and ask the model to retrieve it. Measure recall at each depth. Most models degrade meaningfully past 50-70% of advertised context. Design your workload for the reliable window, not the ceiling.