Streaming Errors Across All AI Providers — 2026 Guide
Streaming responses are how modern AI applications feel fast — but they fail in ways non-streaming requests don't. Mid-stream disconnects leave partial output. Chunks arrive out of order. Timeouts don't behave like request timeouts. Every provider handles streaming differently, with different chunk formats and different failure modes. This is the cross-provider directory of streaming errors.
Streaming responses are how modern AI applications feel fast — but they fail in ways non-streaming requests don't. Mid-stream disconnects leave partial output. Chunks arrive as delta JSON that must be accumulated before parsing. Tool call arguments arrive as fragmentary JSON that will crash any premature parser. Reasoning models have a long silent pause before first token. Content filters run post-stream. Every provider handles streaming differently, with different chunk formats and different failure modes. This page is the cross-provider directory: every streaming failure pattern we've documented across every provider, IDE tool, and gateway, grouped so you can find your specific error in seconds.
For related failure modes, see the Rate Limits, Authentication, Context Window, Timeouts, and Content Filter category hubs. For the underlying architectural patterns, see the Rate Limits pillar guide.
2026 streaming formats at a glance
Streaming formats differ meaningfully across providers — SSE with typed events (Claude), SSE with delta terminator (OpenAI), JSON array chunks (Gemini), model-specific event streams (Bedrock). Here's the 2026 landscape at a glance.
| Provider | Format | Terminator | Keep-alive | Key events |
|---|---|---|---|---|
| Anthropic Claude | SSE with JSON events | message_stop event | ping event every ~15s | message_start, content_block_delta, message_delta, message_stop |
| Anthropic Claude (tool use) | SSE with partial JSON deltas | message_stop event | ping event | input_json_delta chunks — must reassemble |
| OpenAI Chat Completions | SSE | data: [DONE] | None (client handles) | delta.content, delta.tool_calls chunks |
| OpenAI Responses API (2026) | SSE | event: done | None (client handles) | response.output_item.done, response.completed |
| Google Gemini | JSON array chunks | Final chunk with finishReason | None (short timeouts) | candidates[0].content.parts + finishReason |
| AWS Bedrock InvokeModelWithResponseStream | Model-specific event stream | Model-specific (e.g. message_stop for Claude) | None | boto3 iterator, chunk format varies per model |
| AWS Bedrock ConverseStream | Normalized event stream | messageStop event | None | contentBlockDelta, messageStop, metadata |
| Azure OpenAI (Front Door) | SSE | data: [DONE] | None | Same as OpenAI + 60s Front Door cap |
| Azure OpenAI (Direct endpoint) | SSE | data: [DONE] | None | Bypasses Front Door for long streams |
| Gateways (LiteLLM/Portkey) | Normalized SSE | Normalized terminator | None (client handles) | stream_timeout separate from request_timeout |
Anthropic Claude
Claude uses SSE with typed JSON events — message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop. Ping events keep the connection alive every ~15 seconds. Streaming with tool use is the trickiest pattern: tool arguments arrive as input_json_delta chunks that you must accumulate and parse only once complete. Extended thinking events stream separately from regular content blocks.
- Streaming response dropped mid-generationSSE connection lost during output
- Streaming tool use input_json_delta invalid JSONPartial JSON parsed too early
- Streaming extended thinking events not in content blocksThinking events separate stream
- Streaming ping event crashes custom parserping is a valid SSE event, not error
- message_stop event missed — response incompleteClient closed before terminator
OpenAI
OpenAI streams SSE with delta chunks. Chat Completions terminates with data: [DONE]. The Responses API (launched 2026) uses event-typed streaming with event: done as terminator. Streaming with structured outputs enforces the JSON schema at the end — chunks may look invalid until the final chunk. Streaming with reasoning models: reasoning tokens don't stream visibly, so there's a long pause before the first visible content chunk arrives.
- Streaming SSE disconnected mid-responseClient timeout on chunk read
- Streaming tool_call arguments delta returns partial JSONAccumulate all deltas before parsing
- Responses API streaming event: done not receivedStream ended without terminator
- Reasoning model streaming — long pause before first tokenReasoning phase silent
- Structured output streaming — partial JSON parse errorFull response required for schema validation
Google Gemini
Gemini's streamGenerateContent returns JSON chunks (not SSE) with a candidates array. The finishReason field arrives in the final chunk — if it's SAFETY, the stream ended because of content filtering; you may already have partial output that violates the same filter. No keep-alive, so slow generators can time out client-side.
- streamGenerateContent connection drop during generationNo keep-alive from provider
- Streaming finishReason: SAFETY — partial output receivedDiscard partial content, log incident
- Vertex AI vs AI Studio streaming chunk format differsResponse wrapper structure varies
- Streaming with File API attachment — slow first-chunk latencyFile processing precedes streaming
AWS Bedrock
Bedrock offers two streaming APIs: InvokeModelWithResponseStream (model-native format — chunk structure varies per model) and ConverseStream (normalized format across models). The critical Bedrock streaming gotcha: boto3's read_timeout is ignored on streaming responses. The iterator can hang indefinitely if the server stops sending events without closing the connection.
- InvokeModelWithResponseStream hangs (read_timeout ignored)Add client-side timeout wrapper
- InvokeModel stream chunk format varies per modelClaude vs Titan vs Llama
- ConverseStream messageStop event missedNormalized terminator dropped
- Cross-region inference profile streaming latency jitterRegional routing adds variance
Azure OpenAI
Azure OpenAI streaming inherits OpenAI's SSE format but adds the Azure Front Door problem: streams routed through Front Door are capped at 60 seconds regardless of streaming activity. For long-generating responses, use direct-to-deployment endpoints. Content filtering runs post-stream (the whole accumulated response is filtered), so a streamed response can be fully generated then blocked at the terminator.
Streaming at other layers of the stack
Streaming isn't only a provider-layer concern. IDE tools rely on streaming for interactive feel and fail visibly when it breaks. Gateways add their own timeout policies and per-chunk latency overhead. Frameworks wrap streaming in callback protocols or generators that can swallow errors silently.
IDE Tools
IDE tools rely on streaming for interactive feel — Cursor shows code appearing character-by-character, Claude Code streams to terminal, Copilot streams inline suggestions. Streaming failures in IDE tools often manifest as "stuck" progress indicators or partially-inserted code that has to be manually cleaned up.
Gateway Layer
Gateways normalize streaming across providers and add their own timeout policies. LiteLLM has stream_timeout separate from request_timeout — misconfigured gateways can cut streams shorter than the underlying provider would. OpenRouter passes chunks through with additional latency (~25-40ms overhead per chunk cascades on long streams). Portkey captures streaming metrics per chunk for observability.
Framework Layer
Frameworks handle streaming via callback protocols (LangChain), event streams from graph nodes (LangGraph), and generator patterns (LlamaIndex). Framework-layer streaming failures typically involve callback exceptions swallowing errors, generator not properly closing on downstream errors, or streaming being disabled unintentionally by a wrapper component that doesn't support it.