AWS Bedrock Streaming JSON Parse Failed
The Bedrock streaming API works by sending a sequence of events — each event a small JSON object wrapped in an AWS event-stream frame. When the parser breaks mid-stream, the cause is usually a truncated frame, a connection drop, or code that assumes SSE (Server-Sent Events) semantics when Bedrock actually uses AWS binary event streams.
Quick Fix (TL;DR)
Bedrock streams are AWS binary event streams, not SSE — iterate through response["stream"] and handle each typed event (messageStart, contentBlockDelta, contentBlockStop, messageStop, metadata). Use boto3's built-in iterator; don't attempt to parse raw bytes. On EventStreamError, catch, log, and reconnect with the accumulated context. ~50% of streaming parse errors are code assuming SSE; 30% are network drops mid-stream; 20% are edge cases in tool-use streaming.
The Full Error Message
botocore.exceptions.EventStreamError: An error occurred (ModelStreamErrorException) during ConverseStream: Stream terminated before completion due to connection reset.
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) at bedrock_streaming.py:42 in raw_parser
botocore.exceptions.EventStreamError: An error occurred during ConverseStream: Received unexpected event contentBlockStop without matching contentBlockStart.
ConverseStream Event Types
Every ConverseStream response emits these events in this order.
| Event | When It Fires | Contains | Action |
|---|---|---|---|
messageStart | Once at stream open | Assistant role | Initialize accumulator |
contentBlockStart | Start of each content block | Block index + type (text/toolUse) | Track block boundaries |
contentBlockDelta | Repeatedly during block | Text delta or tool-use JSON delta | Append to buffer |
contentBlockStop | End of each content block | Block index | Finalize block |
messageStop | Once at stream close | stopReason (end_turn, tool_use, guardrail_intervened, max_tokens) | Final flush |
metadata | After messageStop | Usage stats (input/output tokens, cache metrics) | Log for billing |
Root Causes (Ranked by Frequency)
response.text, iter_lines, or data: parsing all fail.json.loads each delta fails until the full block completes.contentBlockDelta at index 3 without seeing contentBlockStart at index 3 first — order matters.response["stream"] after the first pass — event streams are single-shot.messageStop has stopReason="guardrail_intervened" that code doesn't handle.read_timeout lower than model max generation time causes premature abort.Fix #1 — Idiomatic ConverseStream Event Loop
The canonical pattern: iterate response["stream"], dispatch each event by its top-level key, accumulate text deltas per content-block index, and handle toolUse JSON assembly at contentBlockStop. This module is the base — copy it into your codebase and extend.
import boto3 import json from botocore.exceptions import EventStreamError, ClientError from botocore.config import Config runtime = boto3.client( "bedrock-runtime", region_name="us-east-1", # Longer read timeout for long generations. config=Config(read_timeout=180, connect_timeout=10), ) def stream_response(model_id: str, messages: list, tools: list = None): """Yield uniform events: {'type': 'text'|'tool_call'|'stop'|'error', ...}.""" request = { "modelId": model_id, "messages": messages, "inferenceConfig": {"maxTokens": 2048}, } if tools: request["toolConfig"] = {"tools": tools} try: resp = runtime.converse_stream(**request) except ClientError as e: yield {"type": "error", "code": e.response["Error"]["Code"], "message": str(e)} return # Per-block accumulators. text_buffers = {} # index -> string tool_use_buffers = {} # index -> {"toolUseId", "name", "input_json"} try: for event in resp["stream"]: # ---- messageStart: mark stream open ---- if "messageStart" in event: continue # ---- contentBlockStart: initialize per-block buffer ---- if "contentBlockStart" in event: blk = event["contentBlockStart"] idx = blk["contentBlockIndex"] start = blk.get("start", {}) if "toolUse" in start: tool_use_buffers[idx] = { "toolUseId": start["toolUse"]["toolUseId"], "name": start["toolUse"]["name"], "input_json": "", } else: text_buffers[idx] = "" # ---- contentBlockDelta: append delta to buffer ---- elif "contentBlockDelta" in event: blk = event["contentBlockDelta"] idx = blk["contentBlockIndex"] delta = blk["delta"] if "text" in delta: text_buffers[idx] = text_buffers.get(idx, "") + delta["text"] yield {"type": "text_delta", "index": idx, "text": delta["text"]} elif "toolUse" in delta: # Tool-use args stream as JSON string fragments. tool_use_buffers[idx]["input_json"] += delta["toolUse"].get("input", "") # ---- contentBlockStop: finalize block ---- elif "contentBlockStop" in event: idx = event["contentBlockStop"]["contentBlockIndex"] if idx in tool_use_buffers: tb = tool_use_buffers.pop(idx) try: parsed_input = json.loads(tb["input_json"] or "{}") except json.JSONDecodeError: parsed_input = {} # partial JSON — model still emitted deltas yield { "type": "tool_call", "tool_use_id": tb["toolUseId"], "name": tb["name"], "input": parsed_input, } # ---- messageStop: final stopReason ---- elif "messageStop" in event: yield { "type": "stop", "reason": event["messageStop"]["stopReason"], } # ---- metadata: usage stats ---- elif "metadata" in event: yield { "type": "metadata", "usage": event["metadata"]["usage"], "metrics": event["metadata"].get("metrics", {}), } except EventStreamError as e: yield {"type": "error", "code": "EventStreamError", "message": str(e)} # Demo consumer if __name__ == "__main__": events = stream_response( "anthropic.claude-sonnet-4-6-20250929-v1:0", [{"role": "user", "content": [{"text": "Count from 1 to 5 slowly."}]}], ) for ev in events: if ev["type"] == "text_delta": print(ev["text"], end="", flush=True) elif ev["type"] == "stop": print(f"\n\n[stopReason={ev['reason']}]") elif ev["type"] == "metadata": print(f"[usage={ev['usage']}]") elif ev["type"] == "error": print(f"\nERROR: {ev['message']}")
Why buffer per index instead of one buffer: a model can emit multiple parallel content blocks (a text block AND a tool-use block in the same response). Indexing by contentBlockIndex keeps them separated so you can render text as it arrives while asynchronously waiting for tool-use blocks to complete.
Fix #2 — Reconnect Strategy for Mid-Stream Drops
Bedrock doesn't offer native stream resume — a dropped connection means starting over. But you can save partial output as context, then request the model to continue from where it stopped. This wrapper implements that pattern.
import boto3 import time from botocore.exceptions import EventStreamError, ClientError runtime = boto3.client("bedrock-runtime", region_name="us-east-1") MAX_ATTEMPTS = 3 def resilient_generate(model_id: str, prompt: str, on_delta) -> str: """Stream to on_delta callback; reconnect with continuation on drop. Returns final accumulated text after all reconnects.""" messages = [{"role": "user", "content": [{"text": prompt}]}] accumulated = "" for attempt in range(MAX_ATTEMPTS): try: resp = runtime.converse_stream( modelId=model_id, messages=messages, inferenceConfig={"maxTokens": 4096}, ) partial = "" complete = False for event in resp["stream"]: if "contentBlockDelta" in event: delta = event["contentBlockDelta"]["delta"] if "text" in delta: partial += delta["text"] on_delta(delta["text"]) elif "messageStop" in event: reason = event["messageStop"]["stopReason"] complete = (reason == "end_turn") accumulated += partial if complete: return accumulated # Stopped for reason other than end_turn (max_tokens, etc.). # Continue with a "please continue" prompt. messages = messages + [ {"role": "assistant", "content": [{"text": accumulated}]}, {"role": "user", "content": [{"text": "Please continue from where you stopped."}]}, ] except EventStreamError as e: print(f"\n[reconnect {attempt+1}/{MAX_ATTEMPTS}: {e}]") if not accumulated: # Never got any text — full retry. time.sleep(2 ** attempt) continue # We have partial output — continue from there. messages = messages + [ {"role": "assistant", "content": [{"text": accumulated}]}, {"role": "user", "content": [{"text": "Connection dropped. Continue from where you left off."}]}, ] time.sleep(2 ** attempt) except ClientError as e: code = e.response["Error"]["Code"] if code in ("ThrottlingException", "ServiceUnavailableException"): time.sleep(2 ** attempt) continue raise return accumulated if __name__ == "__main__": def emit(chunk: str): print(chunk, end="", flush=True) text = resilient_generate( "anthropic.claude-sonnet-4-6-20250929-v1:0", "Write a 500-word explanation of AWS Bedrock streaming.", emit, ) print(f"\n\ntotal length: {len(text)} chars")
Continuation prompt phrasing matters — "Continue from where you left off" works for factual/exposition tasks. For structured output (JSON, code), include the partial text explicitly and instruct: "Complete the following JSON without repeating what's already there:
Fix #3 — Handle Guardrail Intervention Mid-Stream
When a Bedrock Guardrail intercepts output during streaming, the stream ends early with stopReason="guardrail_intervened". Consumer code that expects end_turn as the terminal state renders half a response and thinks it succeeded. This pattern surfaces the intervention to the user.
import boto3 from botocore.exceptions import EventStreamError runtime = boto3.client("bedrock-runtime", region_name="us-east-1") GUARDRAIL_STOP_REASONS = { "guardrail_intervened", "content_filtered", "policy_violation", } def stream_with_guardrail(model_id: str, prompt: str, guardrail_id: str, guardrail_version: str): """Stream with a guardrail attached; surface intervention to caller.""" accumulated = "" intervention = None try: resp = runtime.converse_stream( modelId=model_id, messages=[{"role": "user", "content": [{"text": prompt}]}], guardrailConfig={ "guardrailIdentifier": guardrail_id, "guardrailVersion": guardrail_version, "trace": "enabled", }, inferenceConfig={"maxTokens": 2048}, ) for event in resp["stream"]: if "contentBlockDelta" in event: delta = event["contentBlockDelta"]["delta"] if "text" in delta: accumulated += delta["text"] print(delta["text"], end="", flush=True) elif "messageStop" in event: stop_reason = event["messageStop"]["stopReason"] if stop_reason in GUARDRAIL_STOP_REASONS: intervention = { "reason": stop_reason, "trace": event["messageStop"].get("additionalModelResponseFields", {}), } elif "metadata" in event: # Guardrail trace often appears in metadata. trace = event["metadata"].get("trace", {}) gr_assessment = trace.get("guardrail", {}).get("outputAssessment") if gr_assessment: intervention = intervention or {} intervention["assessment"] = gr_assessment except EventStreamError as e: print(f"\nstream error: {e}") print() if intervention: print(f"[GUARDRAIL INTERVENTION: {intervention['reason']}]") if intervention.get("assessment"): print(f" categories: {intervention['assessment']}") return { "text": accumulated, "was_intervened": intervention is not None, "intervention": intervention, } if __name__ == "__main__": result = stream_with_guardrail( "anthropic.claude-sonnet-4-6-20250929-v1:0", "Give me general information about network security.", "abcd1234efgh", # your guardrail ID "DRAFT", # or a version number ) if result["was_intervened"]: print("Content was moderated. Reasoning above.") else: print(f"Clean output ({len(result['text'])} chars).")
UX pattern for intervention: render the partial text with a subtle indicator (grayed-out or a small notice) that says "This response was cut short by content moderation." Don't pretend the response was complete, and don't retry silently — the user should know moderation happened so they can rephrase.
Prevention Checklist
- Always use boto3's built-in event stream iterator — never try to parse the raw HTTP response body.
- Set
read_timeoutto at least 3x your expected max generation time (default 60s is too short for long outputs). - Buffer text per
contentBlockIndex, not per event — a single response can have multiple parallel content blocks. - For tool-use streaming, only
json.loadsthe input atcontentBlockStop, never on individual deltas — the JSON is incomplete until then. - Always handle
stopReasonbeyondend_turn—max_tokens,tool_use,guardrail_intervened,content_filteredall require different downstream logic. - Log the
metadataevent's usage stats — it's the only reliable way to get final token counts for streaming. - For long-generation workloads, implement continuation resume (Fix #2) rather than treating drops as fatal.
Tested by
Frequently Asked Questions
AWS event streams predate SSE's popularity in the AI space. AWS uses a binary framing format (prelude, headers, payload, CRC) that supports typed multi-event streams and per-event headers — a superset of what SSE offers.
The tradeoff: you must use an AWS SDK to parse them. If you're proxying Bedrock through your own API, translate binary event streams to SSE at the proxy so browser clients can use EventSource.
The metadata event at the end of the stream contains usage — inputTokens, outputTokens, cacheReadInputTokens, cacheWriteInputTokens. This is the same shape as the non-streaming Converse response's usage field.
If you don't see metadata in your event loop, you're probably filtering it out or breaking early — always drain the iterator to end.
invoke_model_with_response_stream provides streaming for legacy InvokeModel, but events are in each provider's native format (see invoke-model-legacy-payload page for the format reference).
converse_stream is the recommended path — same provider-uniform schema as Converse. New code should use ConverseStream; migrate legacy invoke_model_with_response_stream code opportunistically.
Almost always a code bug — the iterator was partially consumed by an earlier pass, or thread-shared state got out of sync.
Bedrock's server always emits contentBlockStart before contentBlockStop for the same index; if you see this error, look for a place your code is either double-iterating the stream or sharing accumulator state across concurrent streams.
The stream stays open as long as the model is generating; there's no hard maximum. But practical limits are set by intermediate proxies — ALB defaults to 60s idle timeout, CloudFront to 30s.
For long generations (30+ seconds), configure your proxy timeouts up to 5+ minutes and set boto3 read_timeout to match. For very long generations (10+ minutes), consider batching multiple shorter Converse calls with continuation logic.
Related Errors
Never get blindsided by an AI provider error again.
Every Tuesday, get the week's newest AWS Bedrock, Anthropic, OpenAI, and Gemini errors — with tested fixes — delivered to your inbox. Free, no spam, unsubscribe with one click.