AI API Response Parsing Errors — JSON, Encoding, SSE Fix Guide 2026 | AI Error Hub

AI Error Hub/Categories/Response Parsing & Encoding Errors

Category Hub

Response Parsing & Encoding Errors

JSON parse, UTF-8 encoding, SSE frame errors — the fix for each.

16 documented errors

Parsing errors sit between the network and your application. The provider sent bytes; you got bytes; something went wrong turning them into usable values. The failures range from obvious (invalid JSON) to subtle (unicode normalization mismatches that pass validation but break downstream comparisons). This hub is every documented response-parsing error and its fix — the layer above network errors and below structured output validation.

Errors in this category

JSON parsing

unexpected_token_json

JSON.parse throws on unexpected token; usually markdown fences or preamble around JSON.

unexpected_end_of_json_input

Response cut off mid-JSON; check max_tokens or read timeout.

invalid_control_character_json

Response contains raw control characters (0x00-0x1F) that aren't escaped.

trailing_comma_json

Model output includes trailing commas; use JSON5 or repair before parsing.

json_comments_not_supported

Response includes // or /* */ comments; standard JSON doesn't allow them.

Encoding & unicode

invalid_utf8_encoding

Response bytes don't decode as valid UTF-8; usually charset mismatch.

unicode_normalization_mismatch

String comparison fails because same visual character has multiple unicode forms.

surrogate_pair_split

Emoji or wide characters split across stream chunks, corrupting the character.

bom_in_response_body

Response includes UTF-8 BOM (byte order mark) that some parsers reject.

Streaming (SSE) parsing

sse_frame_boundary_error

Server-Sent Events frame split incorrectly; missing double-newline separator.

sse_data_parse_error

SSE data field content isn't the expected JSON shape.

sse_event_type_missing

Expected event type field is missing from stream frame.

sse_retry_directive_invalid

SSE retry directive has non-numeric value; parser rejects it.

Compression & content-encoding

gzip_decode_error

Response content-encoding: gzip but body isn't valid gzip stream.

brotli_decode_error

Response content-encoding: br but body isn't valid brotli.

content_length_mismatch

Content-Length header doesn't match actual body bytes received.

Related pillar guides

Frequently asked questions

Why does JSON.parse fail on responses I can see are valid JSON?

Almost always: hidden characters. Markdown fences (```json ... ```), BOM at the start (0xEF 0xBB 0xBF), invisible control characters, or trailing whitespace with a preamble. Strip these before parsing. See G8 Chapter 6 for the repair patterns.

What's the difference between UTF-8 and unicode normalization?

UTF-8 is how characters encode as bytes. Unicode normalization is how the same visual character is represented (é can be one codepoint or 'e' + combining acute accent). Same-looking strings can compare unequal. Fix: normalize both sides with normalize('NFC') before comparing.

Why do emoji sometimes come through corrupted in streaming responses?

Emoji are often 4-byte UTF-8 sequences. When they land on chunk boundaries during streaming, the naive concat can split the sequence. Fix: use TextDecoder with stream: true option (JavaScript) or io.TextIOWrapper with careful buffering (Python).

How do I debug SSE parsing errors?

Log raw bytes as they arrive (before any parsing). Look for missing \n\n separators between events, malformed 'data:' prefixes, or partial frames at connection end. Common cause: proxy stripping or reformatting SSE stream.

Should I decompress responses in my code or let the HTTP client handle it?

Let the HTTP client handle it. Node's http, Python's requests, and browser fetch all handle gzip/brotli transparently by default. Only decompress manually if you're building a low-level client or if the client's default is disabled.

What causes Content-Length mismatches?

Common causes: proxy modified the body (added logging preamble, stripped headers), transfer-encoding switched from length-prefixed to chunked and back, or connection dropped before full body arrived. The last is the most common — treat as a retry-worthy transient failure.

Are parsing errors from the provider or from my code?

Both are possible. Provider issues: content-encoding mismatch, missing headers, malformed SSE frames. Client issues: wrong charset assumption, encoding mismatch, race conditions in chunk assembly. Log raw bytes to isolate.