Structured Output Reliability — Zero JSON Errors

What structured output means and why it fails

Structured output is the pattern where you ask an LLM for a specific data shape — a JSON object, a typed record, a schema-conforming value — instead of free-form text. Every non-trivial AI app relies on it: extracting entities from documents, classifying support tickets into a taxonomy, populating a database row, returning tool arguments the caller can dispatch. When it works, LLMs feel like reliable APIs. When it breaks, your pipeline crashes on malformed JSON, missing required fields, or values that pass parsing but violate business rules.

This guide is about making it work reliably. Not "usually works, sometimes needs a retry." Consistently, at production scale, across every provider, with the tools and patterns that catch failures before your database does.

The four ways structured output can fail

Every structured output failure falls into one of four categories, and each one has a different fix:

The 2026 landscape

Every major provider now offers first-class structured output support. What changed between 2023 and 2026: instead of prompting "respond in JSON" and hoping, you can now attach a JSON schema and the provider guarantees the response conforms syntactically. Semantic and grounding failures still exist, but syntactic and schema failures largely go away when you use the provider primitives correctly.

What this guide covers

Chapter 2 covers the four canonical approaches (JSON mode, schema constraints, tool-as-schema, prefill/grammars) and when to use each. Chapter 3 covers 2026 provider capabilities in depth. Chapter 4 covers schema design — the schemas that models can actually follow well. Chapter 5 covers validation with Pydantic and Zod. Chapter 6 covers failure detection. Chapter 7 covers repair patterns. Chapter 8 covers multi-step and nested extraction. Chapter 9 covers streaming. Chapter 10 covers testing. Chapter 11 covers the 10 common bugs. Chapter 12 is a production-ready helper.

The relationship to function calling (G7)

Function calling and structured output are the same problem viewed from different angles. Function calling says "call this function with these arguments"; structured output says "return this data shape." The underlying model machinery is often identical — providers use their tool-use infrastructure to power structured outputs. What this guide covers that G7 doesn't: cases where you want the data itself rather than an action, streaming structured responses, and multi-schema pipelines.

The four canonical approaches to structured output

Every implementation of structured output is one of four patterns. Each has a sweet spot; each has failure modes when misapplied.

Approach 1: Prompt-only (the naive baseline)

// Just ask nicely
const response = await client.messages.create({
  model: "claude-sonnet-4-6",
  max_tokens: 500,
  messages: [{
    role: "user",
    content: `Extract the person's name and age from this text as JSON:
    "John Smith is 34 years old and lives in Portland."
    
    Respond ONLY with valid JSON matching:
    { "name": "string", "age": number }`,
  }],
});

// Then parse the text and hope for the best
const parsed = JSON.parse(response.content[0].text);

Failure rate: 5-20% depending on model, prompt quality, and edge cases. Common failures: markdown code fences (```json ... ```), preamble text ("Here's the JSON:"), trailing commas, and hallucinated fields. Only appropriate for prototypes.

Approach 2: JSON mode

Providers offer a response_format: { type: "json_object" } flag that guarantees the output will be parseable JSON. The model still chooses the shape; you get syntactic validity for free.

// OpenAI JSON mode
const response = await openai.chat.completions.create({
  model: "gpt-5.6",
  response_format: { type: "json_object" },
  messages: [
    {
      role: "system",
      content: "Respond in JSON with keys 'name' (string) and 'age' (number).",
    },
    {
      role: "user",
      content: "John Smith is 34 years old.",
    },
  ],
});

const parsed = JSON.parse(response.choices[0].message.content!);

Failure rate: ~0% syntactic (JSON always parses), 2-10% schema (wrong shape). No enforcement of your specific structure — just of "some JSON."

Approach 3: JSON schema (structured outputs)

Attach a JSON schema; the provider guarantees the response conforms to it exactly. This is the modern default. Available on OpenAI (response_format: json_schema), Anthropic (via structured output API in 2026), and Google (via response_schema).

// OpenAI structured outputs
const response = await openai.chat.completions.create({
  model: "gpt-5.6",
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "person",
      strict: true,
      schema: {
        type: "object",
        properties: {
          name: { type: "string" },
          age: { type: "integer", minimum: 0, maximum: 150 },
        },
        required: ["name", "age"],
        additionalProperties: false,
      },
    },
  },
  messages: [{
    role: "user",
    content: "John Smith is 34 years old.",
  }],
});

// Guaranteed to conform to schema
const parsed = JSON.parse(response.choices[0].message.content!);

Failure rate: ~0% syntactic + schema. Semantic errors still possible (age=999 if the source text says so). This is what production apps use.

Approach 4: Tool-as-schema

Use the function calling API as a structured output channel. Instead of asking for a tool to be called, you define a tool whose "arguments" are the structured data you want. When the model calls the tool, you get schema-conforming arguments — effectively structured output with the same guarantees as function calling.

// Anthropic: use a tool as your schema
const response = await anthropic.messages.create({
  model: "claude-sonnet-4-6",
  max_tokens: 1024,
  tools: [{
    name: "record_person",
    description: "Record a person's information",
    input_schema: {
      type: "object",
      properties: {
        name: { type: "string" },
        age: { type: "integer", minimum: 0, maximum: 150 },
      },
      required: ["name", "age"],
    },
  }],
  tool_choice: { type: "tool", name: "record_person" },  // force the tool
  messages: [{
    role: "user",
    content: "John Smith is 34 years old.",
  }],
});

// Extract from tool_use block
const toolUse = response.content.find(
  (b) => b.type === "tool_use"
) as any;
const parsed = toolUse.input;  // schema-conforming object

Failure rate: ~0% syntactic + schema when tool_choice forces the tool. This is the classic pattern for Anthropic pre-2026 and still widely used.

Approach 5 (bonus): Prefilling assistant response

An Anthropic-specific trick: prefill the assistant's response with { to force JSON output. Combines with any of the above for extra reliability.

const response = await anthropic.messages.create({
  model: "claude-sonnet-4-6",
  max_tokens: 500,
  messages: [
    {
      role: "user",
      content: "Extract name and age as JSON: John Smith is 34.",
    },
    {
      role: "assistant",
      content: "{",  // prefill forces JSON continuation
    },
  ],
});

const parsed = JSON.parse("{" + response.content[0].text);

Which approach to use

Chapter 3 walks through 2026 provider capabilities in depth, including where each provider draws the line on schema complexity.

Provider capabilities in 2026

Not all structured output APIs are equal. Each provider has different limits, features, and quirks. This chapter is the reference matrix.

Anthropic (Claude)

// 2026 Anthropic structured output
const response = await anthropic.messages.create({
  model: "claude-sonnet-4-6",
  max_tokens: 1024,
  system: "Extract structured data from text.",
  messages: [{ role: "user", content: "..." }],
  // Structured output via tool-as-schema (still the pattern)
  tools: [{
    name: "extract",
    description: "Extract structured info",
    input_schema: yourJSONSchema,
  }],
  tool_choice: { type: "tool", name: "extract" },
});

Capabilities:

• Full JSON Schema draft 2020-12 support in input_schema
• Nested objects, arrays, enums, unions (anyOf, oneOf)
• Regex patterns via pattern keyword
• Numeric ranges via minimum, maximum
• String length constraints via minLength, maxLength

Limits:

• Recursive schemas: supported up to reasonable depth (~10 levels)
• Schema size: keep under ~5,000 tokens for stability
• Additional properties: not strictly forbidden unless you specify additionalProperties: false

OpenAI (GPT)

// 2026 OpenAI structured outputs (native)
const response = await openai.chat.completions.create({
  model: "gpt-5.6",
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "person",
      strict: true,     // enforce schema strictly
      schema: yourJSONSchema,
    },
  },
  messages: [...],
});

Capabilities:

• Native response_format: json_schema with strict: true
• Structured outputs work with both chat completions and Assistants API
• Compatible with function calling: pass structured tools with strict schemas

Limits (as of 2026):

• additionalProperties: false required on every object in strict mode
• All properties must be listed as required in strict mode (use unions with null for optional)
• Max nesting depth: 5 levels
• Max properties per object: 100
• Enums: string only, max 500 values
• Recursive schemas: supported via $ref

Google (Gemini)

// 2026 Google Gemini structured output
const response = await gemini.models.generateContent({
  model: "gemini-3.5-pro",
  contents: [{ role: "user", parts: [{ text: "..." }] }],
  config: {
    responseMimeType: "application/json",
    responseSchema: {
      type: SchemaType.OBJECT,
      properties: {
        name: { type: SchemaType.STRING },
        age: { type: SchemaType.INTEGER },
      },
      required: ["name", "age"],
    },
  },
});

Capabilities:

• responseMimeType: "application/json" guarantees valid JSON
• responseSchema enforces shape
• Uses a subset of OpenAPI 3.0 schema syntax (not full JSON Schema)
• Enum support via string enum arrays

Limits:

• OpenAPI 3.0 subset means some JSON Schema features unavailable (no oneOf, limited anyOf)
• Recursive schemas: limited support
• Constraints (minLength, pattern) supported but enforcement varies by model

The compatibility matrix

Feature                    | Anthropic | OpenAI | Gemini
---------------------------|-----------|--------|--------
JSON syntactic guarantee   |    Yes    |  Yes   |  Yes
Strict schema enforcement  |    Yes    |  Yes   |  Yes
oneOf / discriminated unions|    Yes    |  Yes   |  Limited
Recursive schemas          |    Yes    |  Yes   |  Limited
Regex patterns             |    Yes    |  Yes   |  Yes
Number ranges              |    Yes    |  Yes   |  Yes
Enums                      |    Yes    |  Yes   |  Yes
Max nesting depth          |    ~10    |   5    |   ~5
Additional properties      |  Optional | Forbidden* | Optional
Recursive refs             |    Yes    |  Yes   |  Limited

* OpenAI strict mode requires additionalProperties: false

The portability strategy

If you might switch providers, design your schemas to the smallest common denominator: OpenAI strict mode. If your schemas work with OpenAI strict, they'll work everywhere.

// Portable schema (works on all three)
const personSchema = {
  type: "object",
  properties: {
    name: { type: "string" },
    age: { type: "integer" },
    // Use union with null for optional (OpenAI strict-compatible)
    email: { type: ["string", "null"] },
  },
  required: ["name", "age", "email"],
  additionalProperties: false,
};

Provider-specific gotchas

Anthropic: additionalProperties defaults

Without additionalProperties: false, the model may return extra fields. Usually harmless but can pollute downstream code that iterates over keys. Always set it explicitly.

OpenAI: all-required is strict-mode's cost

In strict mode, every property must be in required. To model optional fields, use type unions with null: { "type": ["string", "null"] }. Verbose but reliable.

Gemini: no oneOf

Gemini's OpenAPI 3.0 subset omits oneOf. If you need discriminated unions, use a single object with an optional discriminator field and validate client-side.

Designing schemas that models can actually follow

A schema that validates is not the same as a schema that produces useful output. The best structured output implementations use schemas designed for the model's cognitive limits, not just for machine-readability. This chapter covers the design principles.

Principle 1: Descriptions are prompts

Every field in a JSON schema can have a description. Models read these. A well-written description is the difference between "email" being filled correctly and being filled with a phone number that looks vaguely email-shaped.

// BAD — no descriptions, model guesses meaning
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "priority": { "type": "string", "enum": ["low", "med", "high"] },
    "assignee": { "type": "string" },
  },
}

// GOOD — descriptions guide extraction
{
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "description": "The task title as it should appear on the board. Maximum 60 characters.",
    },
    "priority": {
      "type": "string",
      "enum": ["low", "med", "high"],
      "description": "Priority based on time-sensitivity: 'high' for urgent (blockers), 'med' for this week, 'low' for backlog.",
    },
    "assignee": {
      "type": "string",
      "description": "The person's first name only, matching one of: Alice, Bob, Carol. Use 'unassigned' if the source doesn't specify.",
    },
  },
}

Principle 2: Prefer enums to free-form strings

Every string field in your schema is a chance for the model to invent values. Enums remove that possibility.

// BAD — model can return anything
"status": { "type": "string" }

// GOOD — bounded set
"status": {
  "type": "string",
  "enum": ["draft", "in_review", "approved", "rejected"],
  "description": "The document's current workflow state."
}

Principle 3: Bound your numbers

// BAD — infinite range
"score": { "type": "integer" }

// GOOD — bounded and documented
"score": {
  "type": "integer",
  "minimum": 0,
  "maximum": 100,
  "description": "Confidence score from 0 (no confidence) to 100 (certain)."
}

Principle 4: Use patterns for structured strings

// Email
"email": {
  "type": "string",
  "pattern": "^[^@\\s]+@[^@\\s]+\\.[^@\\s]+$"
}

// ISO date
"date": {
  "type": "string",
  "pattern": "^\\d{4}-\\d{2}-\\d{2}$",
  "description": "Date in YYYY-MM-DD format."
}

// Phone (E.164)
"phone": {
  "type": "string",
  "pattern": "^\\+[1-9]\\d{6,14}$",
  "description": "International phone in E.164 format, e.g. +14155551234."
}

Principle 5: Make required fields actually required

Every field the model can leave out is a field it will leave out sometimes. If a field is critical, mark it required. If it's optional, model it as a union with null (for OpenAI strict) or omit from required (elsewhere).

// BAD — nothing required; model may skip anything
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "type": "integer" },
  },
}

// GOOD — required fields locked in
{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "age": { "type": "integer", "minimum": 0, "maximum": 150 },
  },
  "required": ["name", "age"],
  "additionalProperties": false,
}

Principle 6: Optional fields as nullable unions (portability)

// Anthropic / non-strict — omit from required
{
  "properties": {
    "email": { "type": "string" },
  },
  "required": ["name"],  // email is optional
}

// OpenAI strict — nullable union
{
  "properties": {
    "email": { "type": ["string", "null"] },
  },
  "required": ["name", "email"],  // email required to be present, may be null
}

Principle 7: Use anyOf/oneOf for discriminated unions

// Event that's either a click or a purchase
{
  "type": "object",
  "oneOf": [
    {
      "properties": {
        "type": { "const": "click" },
        "element_id": { "type": "string" },
      },
      "required": ["type", "element_id"],
    },
    {
      "properties": {
        "type": { "const": "purchase" },
        "amount": { "type": "number", "minimum": 0 },
        "currency": { "type": "string", "enum": ["USD", "EUR", "GBP"] },
      },
      "required": ["type", "amount", "currency"],
    },
  ],
}

Principle 8: Flatten deeply nested schemas

Deep nesting makes it hard for the model to track context. Prefer flat objects when possible.

// BAD — 4 levels deep
{
  "customer": {
    "profile": {
      "personal": {
        "identifiers": {
          "email": { "type": "string" }
        }
      }
    }
  }
}

// GOOD — flat
{
  "customer_email": { "type": "string" }
}

Principle 9: Order matters in properties

Models process JSON left to right during generation. If field B depends on field A, put A first. This is a subtle but real effect.

// BAD — analysis comes before the data it analyzes
{
  "properties": {
    "sentiment_score": { "type": "number" },  // depends on text
    "text": { "type": "string" },
  }
}

// GOOD — data first, derived fields last
{
  "properties": {
    "text": { "type": "string" },
    "sentiment_score": { "type": "number" },
  }
}

Principle 10: Use reasoning fields when analysis is needed

For non-trivial classifications or analyses, add a "reasoning" or "explanation" field before the answer. The model uses it to think through the answer.

{
  "type": "object",
  "properties": {
    "reasoning": {
      "type": "string",
      "description": "Brief explanation of the classification, considering key features."
    },
    "category": {
      "type": "string",
      "enum": ["bug", "feature_request", "question", "spam"],
    },
    "confidence": {
      "type": "number",
      "minimum": 0,
      "maximum": 1,
    },
  },
  "required": ["reasoning", "category", "confidence"],
  "additionalProperties": false,
}

The reasoning field is chain-of-thought embedded in the schema. Discard it after processing if you don't need it — the value is in the model producing it, not in the caller consuming it.

Validation with Pydantic and Zod

Even with schema-enforced generation, you validate on the client. Providers can have bugs; schemas can be misinterpreted; and semantic checks are yours. This chapter covers the two dominant validation libraries.

Pydantic (Python)

from pydantic import BaseModel, Field, EmailStr
from typing import Literal

class Person(BaseModel):
    name: str = Field(min_length=1, max_length=100)
    age: int = Field(ge=0, le=150)
    email: EmailStr
    status: Literal["active", "inactive", "pending"]
    tags: list[str] = Field(default_factory=list, max_length=10)


# From LLM response
from anthropic import Anthropic
client = Anthropic()

def extract_person(text: str) -> Person:
    response = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=500,
        tools=[{
            "name": "record_person",
            "input_schema": Person.model_json_schema(),
        }],
        tool_choice={"type": "tool", "name": "record_person"},
        messages=[{"role": "user", "content": text}],
    )
    
    tool_use = next(b for b in response.content if b.type == "tool_use")
    # Pydantic validates; raises ValidationError if invalid
    return Person(**tool_use.input)

Pydantic advantages

• Types translate directly to JSON schemas via .model_json_schema()
• Runtime validation with clear error messages
• Type-checkable Python code downstream
• Rich built-in types: EmailStr, HttpUrl, PositiveInt, etc.
• Custom validators via @field_validator

Custom validators

from pydantic import BaseModel, Field, field_validator

class Ticket(BaseModel):
    title: str
    priority: Literal["low", "medium", "high"]
    tags: list[str]
    
    @field_validator("tags")
    @classmethod
    def normalize_tags(cls, v: list[str]) -> list[str]:
        # Lowercase and dedupe
        return sorted(set(t.lower().strip() for t in v))
    
    @field_validator("title")
    @classmethod
    def title_not_placeholder(cls, v: str) -> str:
        if v.lower() in ("untitled", "no title", "n/a"):
            raise ValueError("Title cannot be placeholder text")
        return v

Zod (TypeScript)

import { z } from "zod";

const PersonSchema = z.object({
  name: z.string().min(1).max(100),
  age: z.number().int().min(0).max(150),
  email: z.string().email(),
  status: z.enum(["active", "inactive", "pending"]),
  tags: z.array(z.string()).max(10).default([]),
});

type Person = z.infer<typeof PersonSchema>;


// Convert to JSON Schema for the API
import { zodToJsonSchema } from "zod-to-json-schema";

async function extractPerson(text: string): Promise<Person> {
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 500,
    tools: [{
      name: "record_person",
      input_schema: zodToJsonSchema(PersonSchema) as any,
    }],
    tool_choice: { type: "tool", name: "record_person" },
    messages: [{ role: "user", content: text }],
  });
  
  const toolUse = response.content.find((b) => b.type === "tool_use") as any;
  // Zod validates; throws ZodError if invalid
  return PersonSchema.parse(toolUse.input);
}

Zod advantages

• TypeScript-first: z.infer<typeof schema> gives you the type
• Composable: z.union([A, B]), z.discriminatedUnion(...), etc.
• Refinements: .refine((val) => ..., "error message")
• Transforms: .transform((val) => ...) for cleanup
• Converts to JSON Schema via zod-to-json-schema

Custom refinements and transforms

const TicketSchema = z.object({
  title: z.string()
    .min(1)
    .refine(
      (v) => !["untitled", "no title", "n/a"].includes(v.toLowerCase()),
      "Title cannot be placeholder text"
    ),
  priority: z.enum(["low", "medium", "high"]),
  tags: z.array(z.string())
    .transform((tags) => 
      [...new Set(tags.map(t => t.toLowerCase().trim()))].sort()
    ),
});

Vercel AI SDK: generateObject

The Vercel AI SDK ships a generateObject that combines a Zod schema, the provider call, and validation in one function:

import { generateObject } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { z } from "zod";

const { object } = await generateObject({
  model: anthropic("claude-sonnet-4-6"),
  schema: z.object({
    name: z.string(),
    age: z.number().int().min(0).max(150),
    email: z.string().email(),
  }),
  prompt: "Extract from: John Smith, 34, john@example.com",
});

// object is fully typed and validated
console.log(object.name, object.age);

For most TypeScript codebases, this is the shortest path from schema to typed value. See the Vercel AI SDK errors guide for the full pattern.

The validation flow that works

Each layer catches what the previous couldn't. Skipping any of them means production discovers what the layer would have caught.

Handling validation errors

from pydantic import ValidationError

def extract_with_recovery(text: str) -> Person | None:
    try:
        return extract_person(text)
    except ValidationError as e:
        # Get structured error info
        errors = e.errors()
        for err in errors:
            print(f"Field {err['loc']}: {err['msg']}")
        # Chapter 7 covers repair strategies; for now, log and return None
        return None

// TypeScript version
try {
  const person = await extractPerson(text);
} catch (e) {
  if (e instanceof z.ZodError) {
    console.log("Validation errors:");
    for (const issue of e.issues) {
      console.log(`  ${issue.path.join(".")}: ${issue.message}`);
    }
  }
  throw e;
}

Common failure modes and how to detect them

Chapter 1 named four failure categories. This chapter is the field guide — what each looks like in production, how to detect it, and what signals it in your metrics.

Failure mode 1: Markdown code fences around JSON

// The model's response
```json
{"name": "John", "age": 34}
```

// Parsing fails: "Unexpected token '`' at position 0"

Detection: response starts with backticks or "```". Fix: use JSON mode / structured output API; if you can't, strip markdown fences before parsing.

function stripMarkdownFences(text: string): string {
  return text
    .replace(/^```(?:json)?\s*/i, "")
    .replace(/\s*```$/, "")
    .trim();
}

Failure mode 2: Preamble text before the JSON

// Response:
Here is the extracted data:
{"name": "John", "age": 34}

// Parsing fails at "Here"

Detection: response doesn't start with { or [. Fix: use structured output API; if you can't, extract the first JSON object with a regex.

function extractFirstJson(text: string): string | null {
  const match = text.match(/\{[\s\S]*\}|\[[\s\S]*\]/);
  return match ? match[0] : null;
}

Failure mode 3: Trailing commas

// Response:
{"name": "John", "age": 34,}
                          ^^ trailing comma

// Standard JSON.parse fails

Detection: parser throws on trailing comma. Fix: use a lenient parser or strip trailing commas before parsing.

import JSON5 from "json5";

// JSON5 tolerates trailing commas, single quotes, comments
const parsed = JSON5.parse(text);

Failure mode 4: Truncated response

// Response cut off mid-JSON
{"name": "John Smith", "age": 34, "biography": "John was born in 1990 and grew up in

// Ends mid-string; parser fails

Detection: parse fails AND response length is at or very near max_tokens. Fix: increase max_tokens, or split the extraction into smaller chunks.

function isLikelyTruncated(
  response: any,
  maxTokens: number
): boolean {
  const usage = response.usage;
  const outputTokens = usage?.output_tokens ?? usage?.completion_tokens ?? 0;
  // Within 10% of the ceiling
  return outputTokens >= maxTokens * 0.9;
}

Failure mode 5: Wrong type

// Schema says age is a number
{"name": "John", "age": "thirty-four"}
                        ^^^^^^^^^^^^^^ string, not number

// Schema-enforced generation prevents this;
// prompt-only extraction commonly fails here

Detection: Pydantic/Zod validation fails with type coercion error. Fix: upgrade to structured output API; if you can't, add explicit type coercion (with bounds).

Failure mode 6: Missing required fields

// Schema requires name and age
{"name": "John"}
                ^^ age missing

// Validation: "field required"

Detection: schema validation error naming the missing field. Fix: mark field as required in schema; if it's truly optional, model as nullable union or omit from required list.

Failure mode 7: Extra fields (schema drift)

// Model adds fields not in schema
{
  "name": "John",
  "age": 34,
  "biography": "A brief bio the model invented"
}

Detection: object has keys not in schema. Fix: set additionalProperties: false in schema (mandatory in OpenAI strict mode). Some providers still allow drift; validate client-side.

Failure mode 8: Enum violation

// Schema: status ∈ ["active", "inactive", "pending"]
{"status": "on hold"}

// Parses as string but not in enum

Detection: Pydantic Literal or Zod enum validation fails. Fix: use structured output API with strict enums; add fuzzy matching (nearest enum) as a repair step; add a description explaining the intended values.

Failure mode 9: Out-of-range values

// Schema: age is 0-150
{"age": 999}

// Passes shape check; fails semantic

Detection: Pydantic/Zod range validation. Fix: range constraints in schema (minimum, maximum), stronger prompt language, or the reasoning-field pattern from Chapter 4.

Failure mode 10: Hallucinated content

// Source: "Reach out to our support team"
// Model fills:
{"contact_email": "support@example.com"}

// Parses. Validates. But the email was invented.

Detection: hardest one. Passes all technical checks. Detection requires:

• Cross-referencing the value against the source text
• Requiring the model to quote source snippets alongside extractions
• Human review for high-stakes extractions
• Comparing across multiple extraction runs for consistency

// The quote-alongside pattern
const schema = z.object({
  contact_email: z.string().email(),
  contact_email_source_quote: z.string().describe(
    "Verbatim quote from the source text containing this email. If the email is not explicitly stated, return empty string."
  ),
});

// Post-validation: reject if quote is empty
const result = await extractContact(text);
if (!result.contact_email_source_quote.includes(result.contact_email)) {
  throw new Error("Contact email not grounded in source");
}

The detection metric to track

// Per-schema validation failure rate
metrics.increment("extraction.attempt", { schema: "person" });
try {
  const result = PersonSchema.parse(rawInput);
  metrics.increment("extraction.success", { schema: "person" });
  return result;
} catch (e) {
  if (e instanceof z.ZodError) {
    for (const issue of e.issues) {
      metrics.increment("extraction.failure", {
        schema: "person",
        field: issue.path.join("."),
        code: issue.code,
      });
    }
  }
  throw e;
}

Alert when failure rate on any schema exceeds baseline. A jump means the model changed behavior (provider update), your schema drifted from source data reality, or your prompt regressed.

Repair patterns — auto-fix, re-ask, and fallback

When structured output fails validation, you have three options: repair it programmatically, ask the model to fix it, or fall back to a simpler shape. Each is right in different situations.

Pattern 1: Programmatic auto-repair

For syntactic failures with well-known shapes, fix in code before validation.

function repairJSON(raw: string): string {
  let repaired = raw.trim();
  
  // Strip markdown fences
  repaired = repaired.replace(/^```(?:json)?\s*/i, "");
  repaired = repaired.replace(/\s*```$/, "");
  
  // Strip preamble by finding first { or [
  const startMatch = repaired.match(/[\{\[]/);
  if (startMatch && startMatch.index > 0) {
    repaired = repaired.slice(startMatch.index);
  }
  
  // Strip suffix past the matching close
  const closeIdx = findMatchingClose(repaired);
  if (closeIdx !== -1) {
    repaired = repaired.slice(0, closeIdx + 1);
  }
  
  // Fix trailing commas
  repaired = repaired.replace(/,(\s*[\}\]])/g, "$1");
  
  return repaired;
}


// Try structured parse then attempt repair
function tryParse<T>(raw: string, schema: z.ZodType<T>): T | null {
  try {
    return schema.parse(JSON.parse(raw));
  } catch {}
  try {
    return schema.parse(JSON.parse(repairJSON(raw)));
  } catch {}
  return null;
}

Pattern 2: Re-ask the model

For schema violations where the raw text has the information but the shape is wrong, ask the model to fix its own output.

async function extractWithRepair<T>(
  text: string,
  schema: z.ZodType<T>,
  maxAttempts = 3,
): Promise<T> {
  let lastError: unknown;
  let lastResponse: string | null = null;
  
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    const messages: any[] = [
      { role: "user", content: `Extract data from: ${text}` },
    ];
    
    // On retry, show the model its own bad output
    if (lastResponse && lastError instanceof z.ZodError) {
      messages.push(
        { role: "assistant", content: lastResponse },
        {
          role: "user",
          content: `Your previous response failed validation:
${lastError.issues.map(i => `- ${i.path.join(".")}: ${i.message}`).join("\n")}

Fix the errors and respond with valid JSON.`,
        }
      );
    }
    
    const response = await callModelWithSchema(messages, schema);
    lastResponse = response.raw;
    
    try {
      return schema.parse(response.parsed);
    } catch (e) {
      lastError = e;
      if (attempt === maxAttempts - 1) throw e;
    }
  }
  throw lastError;
}

Re-asking works because models are quite good at reading their own errors and fixing them. Typical: 60-80% of first-attempt validation failures fix on the second attempt.

Pattern 3: Fallback to a simpler schema

When repeated attempts fail, degrade gracefully. Extract with a looser schema that captures partial information rather than failing entirely.

const StrictPersonSchema = z.object({
  name: z.string().min(1),
  age: z.number().int().min(0).max(150),
  email: z.string().email(),
  phone: z.string().regex(/^\+[1-9]\d{6,14}$/),
});

const LoosePersonSchema = z.object({
  name: z.string().min(1),
  age: z.number().int().nullable(),
  email: z.string().nullable(),
  phone: z.string().nullable(),
});


async function extractPerson(text: string) {
  try {
    return await extractWithSchema(text, StrictPersonSchema);
  } catch {
    // Fall back to loose extraction
    const loose = await extractWithSchema(text, LoosePersonSchema);
    // Flag record for review
    await queueForReview(loose, "loose_extraction");
    return loose;
  }
}

Pattern 4: Field-level repair

For semantic failures on specific fields, ask the model to fix just those fields.

async function repairField(
  originalText: string,
  currentValue: any,
  field: string,
  error: string,
): Promise<any> {
  const response = await client.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 200,
    messages: [{
      role: "user",
      content: `Original text: "${originalText}"

Current extracted value for '${field}': ${JSON.stringify(currentValue)}
Validation error: ${error}

Provide just the corrected value for '${field}' as JSON.`,
    }],
  });
  
  return JSON.parse(response.content[0].text);
}

Pattern 5: Fuzzy enum matching

When the model returns a value close to an enum but not exact, snap to the nearest match.

function snapToEnum(value: string, allowed: string[]): string | null {
  const normalized = value.toLowerCase().trim();
  
  // Exact match
  for (const a of allowed) {
    if (a.toLowerCase() === normalized) return a;
  }
  
  // Contains match
  for (const a of allowed) {
    if (normalized.includes(a.toLowerCase()) || 
        a.toLowerCase().includes(normalized)) {
      return a;
    }
  }
  
  // Levenshtein distance under threshold
  let best: string | null = null;
  let bestDist = Infinity;
  for (const a of allowed) {
    const dist = levenshtein(normalized, a.toLowerCase());
    if (dist < bestDist && dist <= 2) {
      best = a;
      bestDist = dist;
    }
  }
  return best;
}


// Use in Zod transform
const PrioritySchema = z.string().transform((v, ctx) => {
  const snapped = snapToEnum(v, ["low", "medium", "high"]);
  if (!snapped) {
    ctx.addIssue({ code: "custom", message: `Unknown priority: ${v}` });
    return z.NEVER;
  }
  return snapped;
});

Pattern 6: The repair pipeline

Composing all the patterns into a coherent flow:

async function extract<T>(
  text: string,
  schema: z.ZodType<T>,
  fallbackSchema?: z.ZodType<any>,
): Promise<T | any> {
  // Step 1: Try with schema-enforced generation
  try {
    const raw = await callProviderWithSchema(text, schema);
    return schema.parse(raw);
  } catch (e) { /* proceed */ }
  
  // Step 2: Try programmatic repair on the raw response
  try {
    const raw = await callProviderPlain(text);
    const repaired = repairJSON(raw);
    return schema.parse(JSON.parse(repaired));
  } catch (e) { /* proceed */ }
  
  // Step 3: Re-ask with error feedback (2 attempts)
  try {
    return await extractWithRepair(text, schema, 2);
  } catch (e) { /* proceed */ }
  
  // Step 4: Fall back to looser schema if provided
  if (fallbackSchema) {
    try {
      const loose = await extractWithSchema(text, fallbackSchema);
      await queueForReview(loose, "fallback_used");
      return loose;
    } catch {}
  }
  
  // Step 5: Give up cleanly
  throw new Error("Extraction failed after all repair attempts");
}

When NOT to repair

The cost of aggressive repair

Each repair attempt costs another API call. Track the ratio:

// If your repair rate exceeds 20%, something upstream is wrong.
// Fix the schema, the prompt, or the model — not just the repair loop.

metrics.gauge("extraction.repair_rate", 
  repair_count / total_attempts);

Multi-step extractions and nested schemas

Real extractions rarely fit in one call. Documents have multiple entities. Records have nested substructures. Pipelines chain outputs into further extractions. This chapter covers the patterns.

Multi-entity extraction

Extract a list of records from a single source. The temptation is one big schema with an array; the practice is often better as two calls.

Approach A: One call with an array field

const InvoiceSchema = z.object({
  invoice_number: z.string(),
  date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/),
  line_items: z.array(z.object({
    description: z.string(),
    quantity: z.number().int().min(1),
    unit_price: z.number().min(0),
    total: z.number().min(0),
  })).min(1),
  total_amount: z.number().min(0),
});

const invoice = await extractWithSchema(invoiceText, InvoiceSchema);

Works when: the record count is small (say, under 20 items), context fits comfortably, and all items share the same schema. Fails at scale — long lists inside a single response often get truncated or miss items.

Approach B: Two-pass extraction

// Pass 1: extract headers/summary and count
const InvoiceHeaderSchema = z.object({
  invoice_number: z.string(),
  date: z.string(),
  line_item_count: z.number().int().min(0),
});
const header = await extractWithSchema(text, InvoiceHeaderSchema);


// Pass 2: extract line items in batches
const LineItemSchema = z.object({
  description: z.string(),
  quantity: z.number().int(),
  unit_price: z.number(),
});

async function extractLineItems(text: string, expectedCount: number) {
  const items = [];
  const batchSize = 10;
  for (let start = 0; start < expectedCount; start += batchSize) {
    const batch = await extractWithSchema(text, z.object({
      items: z.array(LineItemSchema).length(
        Math.min(batchSize, expectedCount - start)
      ),
    }), { skipItems: start });
    items.push(...batch.items);
  }
  return items;
}

Works when item counts are large, unpredictable, or when each item's context is substantial.

Two-schema staged extractions

Some extractions have a "wide net" step (find all candidates) and a "deep dive" step (extract detail per candidate).

// Stage 1: identify all people mentioned
const PeopleSchema = z.object({
  people: z.array(z.object({
    name: z.string(),
    first_mention_snippet: z.string(),
  })),
});

const people = await extractWithSchema(document, PeopleSchema);


// Stage 2: for each person, extract detailed profile
const PersonProfileSchema = z.object({
  name: z.string(),
  role: z.string().nullable(),
  organization: z.string().nullable(),
  contact: z.object({
    email: z.string().email().nullable(),
    phone: z.string().nullable(),
  }),
  quoted_statements: z.array(z.string()),
});

const profiles = await Promise.all(
  people.people.map((p) =>
    extractWithSchema(
      document,
      PersonProfileSchema,
      { context: `Focus on: ${p.name}` }
    )
  )
);

Better than one giant schema because:

• Each stage has a simpler task to reason about
• Parallelizable in stage 2
• Failures in one profile don't cascade to others
• Retry logic can target just the failed pieces

Nested schemas — the depth trap

Deep nesting harms model performance. Rule of thumb: 3 levels is comfortable, 5 is the ceiling, deeper is trouble.

// TOO DEEP — 6 levels of nesting
const CompanySchema = z.object({
  company: z.object({
    departments: z.array(z.object({
      teams: z.array(z.object({
        members: z.array(z.object({
          contact: z.object({
            phone_numbers: z.array(z.object({
              number: z.string(),
              type: z.string(),
            })),
          }),
        })),
      })),
    })),
  }),
});

Better: flatten via lookup tables.

// Flattened relational shape
const FlatCompanySchema = z.object({
  company: z.object({
    id: z.string(),
    name: z.string(),
  }),
  departments: z.array(z.object({
    id: z.string(),
    company_id: z.string(),
    name: z.string(),
  })),
  teams: z.array(z.object({
    id: z.string(),
    department_id: z.string(),
    name: z.string(),
  })),
  members: z.array(z.object({
    id: z.string(),
    team_id: z.string(),
    name: z.string(),
  })),
  phone_numbers: z.array(z.object({
    member_id: z.string(),
    number: z.string(),
    type: z.enum(["mobile", "office", "home"]),
  })),
});

Both express the same data. The flat version is easier for models to fill correctly and easier for downstream code to process.

Chain-of-schema reasoning

For complex extractions, use a reasoning schema before the final schema. The model thinks in one call, then extracts in the next.

// Step 1: identify structure and reasoning
const AnalysisSchema = z.object({
  document_type: z.enum(["invoice", "receipt", "contract", "other"]),
  key_entities_mentioned: z.array(z.string()),
  extraction_challenges: z.array(z.string()),
});


// Step 2: use the analysis to guide extraction
const analysis = await extractWithSchema(doc, AnalysisSchema);

const InvoiceSchema = z.object({/*...*/});
const invoice = await extractWithSchema(doc, InvoiceSchema, {
  systemPrompt: `You've already analyzed this document:
${JSON.stringify(analysis, null, 2)}

Now extract the invoice fields, being careful of these challenges:
${analysis.extraction_challenges.join(", ")}`,
});

Discriminated unions for polymorphic data

When entities can be different shapes, use discriminated unions.

const EventSchema = z.discriminatedUnion("type", [
  z.object({
    type: z.literal("page_view"),
    url: z.string().url(),
    referrer: z.string().url().nullable(),
  }),
  z.object({
    type: z.literal("click"),
    element_id: z.string(),
    x: z.number(),
    y: z.number(),
  }),
  z.object({
    type: z.literal("purchase"),
    amount: z.number().positive(),
    currency: z.enum(["USD", "EUR", "GBP"]),
    items: z.array(z.string()),
  }),
]);

The type field is the discriminator; the model chooses which shape and fills accordingly. Both Zod and Pydantic support this pattern cleanly.

Streaming per-item vs whole-response

For long extractions, stream item-by-item to start processing before the whole response finishes. Chapter 9 covers streaming in depth.

Streaming structured output

Streaming isn't just for chat. Long structured extractions benefit from progressive parsing: start rendering item 1 of a list while items 2-10 are still generating. This chapter covers the patterns.

Why stream structured output

• Perceived latency: users see progress instead of a spinner
• Early termination: cancel expensive generation if early results are wrong
• Downstream pipelining: start processing item 1 while item 10 still streams
• Memory efficiency: don't buffer all 500 records before writing to database

The parsing challenge

JSON is not stream-friendly by default. A partial JSON string like {"items": [{"name": "Alice", "age": doesn't parse. You need incremental JSON parsers.

Anthropic streaming with input JSON deltas

When using tool-as-schema with streaming, Anthropic sends the tool input as incremental JSON deltas via input_json_delta events.

const response = await anthropic.messages.stream({
  model: "claude-sonnet-4-6",
  max_tokens: 4096,
  tools: [{
    name: "extract_items",
    input_schema: {
      type: "object",
      properties: {
        items: {
          type: "array",
          items: { type: "object", properties: {
            name: { type: "string" },
            price: { type: "number" },
          }},
        },
      },
      required: ["items"],
    },
  }],
  tool_choice: { type: "tool", name: "extract_items" },
  messages: [{ role: "user", content: "Extract items from: ..." }],
});

// Accumulate the streamed JSON incrementally
let accumulated = "";

for await (const event of response) {
  if (event.type === "content_block_delta" && 
      event.delta.type === "input_json_delta") {
    accumulated += event.delta.partial_json;
    
    // Try to parse the partial as complete objects grow
    const complete = tryExtractCompleteItems(accumulated);
    for (const item of complete) {
      await processItem(item);  // stream downstream
    }
  }
}

Incremental parsing with a partial JSON parser

import { parse } from "partial-json";

function tryExtractCompleteItems(partial: string): any[] {
  try {
    // partial-json returns the largest parseable subset
    const parsed = parse(partial);
    if (parsed?.items && Array.isArray(parsed.items)) {
      // Return items whose structure is complete
      return parsed.items.filter(isCompleteItem);
    }
  } catch {}
  return [];
}


function isCompleteItem(item: any): boolean {
  // Item is complete if it has all required fields
  return item && typeof item.name === "string" && 
         typeof item.price === "number";
}

OpenAI streaming with structured outputs

const stream = await openai.chat.completions.create({
  model: "gpt-5.6",
  stream: true,
  response_format: {
    type: "json_schema",
    json_schema: { name: "items", strict: true, schema: /*...*/ },
  },
  messages: [/*...*/],
});

let content = "";
for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content ?? "";
  content += delta;
  
  const items = tryExtractCompleteItems(content);
  for (const item of items) {
    await processItem(item);
  }
}

Emitting items to the client as they complete

// Server-sent events for streaming extraction results
export async function GET(request: Request) {
  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(controller) {
      const emittedIndices = new Set<number>();
      let accumulated = "";
      
      for await (const event of anthropicStream) {
        if (event.type === "content_block_delta" && 
            event.delta.type === "input_json_delta") {
          accumulated += event.delta.partial_json;
          
          const items = tryExtractCompleteItems(accumulated);
          items.forEach((item, i) => {
            if (!emittedIndices.has(i)) {
              emittedIndices.add(i);
              controller.enqueue(encoder.encode(
                `data: ${JSON.stringify({ index: i, item })}\n\n`
              ));
            }
          });
        }
      }
      controller.close();
    },
  });
  
  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
    },
  });
}

Vercel AI SDK streaming

import { streamObject } from "ai";
import { z } from "zod";

const { partialObjectStream } = await streamObject({
  model: anthropic("claude-sonnet-4-6"),
  schema: z.object({
    items: z.array(z.object({
      name: z.string(),
      price: z.number(),
    })),
  }),
  prompt: "Extract items from: ...",
});

for await (const partial of partialObjectStream) {
  // partial is a partially-typed object as it grows
  if (partial.items) {
    for (const item of partial.items) {
      if (item.name && item.price !== undefined) {
        await processItem(item);
      }
    }
  }
}

The completion detection pattern

You need to know when an item is "done" so you don't process incomplete data. Two approaches:

Approach A: All required fields present

function isComplete(item: Partial<Item>, schema: z.ZodType<Item>): item is Item {
  const result = schema.safeParse(item);
  return result.success;
}

Approach B: Structural completeness (next item started)

// If we're at items[2] and items[3] has any content,
// items[2] must be complete
function isCompleteByPosition(items: any[], index: number): boolean {
  return index < items.length - 1;
}

Approach A is safer but rejects items with valid-but-still-streaming string fields. Approach B is more permissive but only works after the next item starts.

Cancellation during streaming

const controller = new AbortController();

// Timeout after 30s
setTimeout(() => controller.abort(), 30_000);

try {
  const stream = anthropic.messages.stream({
    model: "claude-sonnet-4-6",
    messages: [/*...*/],
    signal: controller.signal,
  });
  
  for await (const event of stream) {
    if (controller.signal.aborted) break;
    // process event
  }
} catch (e) {
  if (e.name === "AbortError") {
    console.log("Extraction cancelled");
  }
}

The pitfall: assuming streaming is atomic

// WRONG — this may emit the same item twice
for await (const event of stream) {
  accumulated += event.delta.partial_json;
  const items = tryExtractCompleteItems(accumulated);
  for (const item of items) {
    await processItem(item);  // BUG: items[0] emits on every iteration
  }
}

// RIGHT — track what's been emitted
const emittedIndices = new Set<number>();
for await (const event of stream) {
  accumulated += event.delta.partial_json;
  const items = tryExtractCompleteItems(accumulated);
  items.forEach((item, i) => {
    if (!emittedIndices.has(i)) {
      emittedIndices.add(i);
      processItem(item);
    }
  });
}

Testing structured output pipelines

Structured output pipelines have three testing surfaces: your schemas, your extraction logic, and end-to-end correctness. Each needs a different approach.

Test 1: Schema correctness

Your schema should reject invalid inputs and accept valid ones. Test the schema directly, without touching the LLM.

import { describe, it, expect } from "vitest";
import { PersonSchema } from "./schemas";

describe("PersonSchema", () => {
  it("accepts valid person", () => {
    const result = PersonSchema.safeParse({
      name: "Alice",
      age: 30,
      email: "alice@example.com",
    });
    expect(result.success).toBe(true);
  });
  
  it("rejects missing required field", () => {
    const result = PersonSchema.safeParse({
      name: "Alice",
      // age missing
      email: "alice@example.com",
    });
    expect(result.success).toBe(false);
  });
  
  it("rejects out-of-range age", () => {
    const result = PersonSchema.safeParse({
      name: "Alice",
      age: 999,
      email: "alice@example.com",
    });
    expect(result.success).toBe(false);
  });
  
  it("rejects invalid email format", () => {
    const result = PersonSchema.safeParse({
      name: "Alice",
      age: 30,
      email: "not-an-email",
    });
    expect(result.success).toBe(false);
  });
  
  it("rejects extra fields when strict", () => {
    const result = PersonSchema.strict().safeParse({
      name: "Alice",
      age: 30,
      email: "alice@example.com",
      extra_field: "surprise",
    });
    expect(result.success).toBe(false);
  });
});

Test 2: Extraction logic with mocked provider

Mock the provider to return specific responses; test that your code handles them correctly.

import { vi } from "vitest";

const mockAnthropic = {
  messages: {
    create: vi.fn(),
  },
};

describe("extractPerson", () => {
  it("parses valid tool response", async () => {
    mockAnthropic.messages.create.mockResolvedValue({
      content: [{
        type: "tool_use",
        name: "record_person",
        input: { name: "Alice", age: 30, email: "alice@ex.com" },
      }],
    });
    
    const result = await extractPerson("Alice is 30, alice@ex.com");
    expect(result).toEqual({
      name: "Alice",
      age: 30,
      email: "alice@ex.com",
    });
  });
  
  it("throws on missing tool_use block", async () => {
    mockAnthropic.messages.create.mockResolvedValue({
      content: [{ type: "text", text: "I couldn't extract that." }],
    });
    
    await expect(extractPerson("garbage input")).rejects.toThrow();
  });
  
  it("throws ZodError on schema violation", async () => {
    mockAnthropic.messages.create.mockResolvedValue({
      content: [{
        type: "tool_use",
        name: "record_person",
        input: { name: "Alice", age: 999, email: "not-email" },
      }],
    });
    
    await expect(extractPerson("...")).rejects.toThrow(z.ZodError);
  });
});

Test 3: Repair logic

describe("repairJSON", () => {
  it("strips markdown fences", () => {
    const input = '```json\n{"name": "Alice"}\n```';
    expect(repairJSON(input)).toBe('{"name": "Alice"}');
  });
  
  it("strips preamble text", () => {
    const input = 'Here is the JSON:\n{"name": "Alice"}';
    expect(repairJSON(input)).toBe('{"name": "Alice"}');
  });
  
  it("removes trailing commas", () => {
    const input = '{"name": "Alice", "age": 30,}';
    expect(repairJSON(input)).toBe('{"name": "Alice", "age": 30}');
  });
  
  it("handles all three at once", () => {
    const input = '```json\nHere\'s the data:\n{"a": 1, "b": 2,}\n```';
    const repaired = repairJSON(input);
    expect(JSON.parse(repaired)).toEqual({ a: 1, b: 2 });
  });
});

Test 4: Golden dataset (evaluation)

The most important type of test for LLM pipelines. Curated input-output pairs; run periodically to measure accuracy.

// tests/golden-extractions.json
{
  "cases": [
    {
      "input": "John Smith is a 34-year-old engineer at Acme Corp.",
      "expected": {
        "name": "John Smith",
        "age": 34,
        "role": "engineer",
        "organization": "Acme Corp",
      },
    },
    {
      "input": "Contact Sarah at sarah@company.io — she's the CTO.",
      "expected": {
        "name": "Sarah",
        "email": "sarah@company.io",
        "role": "CTO",
        "organization": null,
      },
    },
    // ... 50+ cases
  ]
}

import goldenCases from "./tests/golden-extractions.json";

describe("Golden extractions", () => {
  it.each(goldenCases.cases)("extracts case correctly", async (testCase) => {
    const actual = await extractPerson(testCase.input);
    
    // Field-by-field comparison
    for (const [field, expected] of Object.entries(testCase.expected)) {
      expect(actual[field]).toEqual(expected);
    }
  });
});

Test 5: Regression tracking

Run golden tests on every commit and against multiple providers. Track accuracy over time.

// evaluation-runner.ts
async function runEvaluation() {
  const results = {
    total: goldenCases.length,
    passed: 0,
    failedByField: {} as Record<string, number>,
  };
  
  for (const testCase of goldenCases) {
    try {
      const actual = await extractPerson(testCase.input);
      let allMatch = true;
      for (const [field, expected] of Object.entries(testCase.expected)) {
        if (JSON.stringify(actual[field]) !== JSON.stringify(expected)) {
          results.failedByField[field] = (results.failedByField[field] || 0) + 1;
          allMatch = false;
        }
      }
      if (allMatch) results.passed++;
    } catch (e) {
      // extraction failed entirely
    }
  }
  
  const accuracy = results.passed / results.total;
  console.log(`Accuracy: ${(accuracy * 100).toFixed(1)}%`);
  console.log(`Failed fields:`, results.failedByField);
  
  // Emit to metrics
  metrics.gauge("extraction.golden_accuracy", accuracy);
  
  return results;
}

Test 6: Fuzz testing

Generate adversarial inputs to find failure modes.

const adversarialInputs = [
  "",  // empty
  "the",  // single word
  "a".repeat(10000),  // huge
  "🤖👾💾",  // emoji-only
  "SELECT * FROM users; DROP TABLE users;",  // SQL injection-like
  "{'name': 'Alice'}",  // Python-style dict (not JSON)
  "<person>Alice</person>",  // XML
  "이름은 Alice입니다",  // non-Latin
];

describe("Fuzz testing", () => {
  it.each(adversarialInputs)("handles adversarial input: %s", async (input) => {
    // Should either succeed or throw a specific error, never crash
    try {
      const result = await extractPerson(input);
      expect(result).toBeDefined();
    } catch (e) {
      expect(e).toBeInstanceOf(Error);
      // Should not be an unexpected internal error
      expect(e.message).toBeDefined();
    }
  });
});

What NOT to test

The evaluation cadence

# CI/CD integration
# .github/workflows/eval.yml
name: LLM Evaluation
on:
  pull_request:
    paths:
      - "src/extraction/**"
      - "tests/golden-*.json"
  schedule:
    - cron: "0 3 * * *"  # nightly

jobs:
  eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm install
      - run: npm run eval:golden
      - name: Fail if accuracy dropped
        run: |
          ACCURACY=$(cat eval-results.json | jq -r .accuracy)
          if (( $(echo "$ACCURACY < 0.95" | bc -l) )); then
            echo "Accuracy dropped to $ACCURACY"
            exit 1
          fi

10 bugs that cause JSON errors in production

Every one of these is a bug I've seen ship. Together, they cover 90% of structured output failures in production apps.

Bug 1: Not using structured output APIs when available

Prompt-only extraction ("respond in JSON") when the provider supports schema-enforced generation. Failure rate goes from ~0% to 5-20%.

// WRONG — asking politely
const response = await client.messages.create({
  messages: [{ role: "user", content: "Extract as JSON: ..." }],
});
const parsed = JSON.parse(response.content[0].text);

// RIGHT — schema-enforced
const response = await client.messages.create({
  tools: [{ name: "extract", input_schema: schema }],
  tool_choice: { type: "tool", name: "extract" },
  messages: [{ role: "user", content: "..." }],
});

Bug 2: Missing additionalProperties: false

Model adds extra fields; your code either breaks or silently accepts them. On OpenAI strict mode, you must set this; on other providers, you should.

// WRONG — model may add fields
{
  "type": "object",
  "properties": { "name": {"type": "string"} },
  "required": ["name"],
}

// RIGHT
{
  "type": "object",
  "properties": { "name": {"type": "string"} },
  "required": ["name"],
  "additionalProperties": false,
}

Bug 3: Forgetting to instrument validation failures

Silent failure isn't just annoying — it's expensive when you find out about it from customer complaints instead of your dashboard.

// WRONG — try/catch that swallows errors
try {
  return schema.parse(raw);
} catch {
  return null;  // no visibility
}

// RIGHT — instrument the failure
try {
  return schema.parse(raw);
} catch (e) {
  if (e instanceof z.ZodError) {
    for (const issue of e.issues) {
      metrics.increment("schema.failure", {
        schema: "person",
        field: issue.path.join("."),
        code: issue.code,
      });
    }
  }
  throw e;
}

Bug 4: Field descriptions omitted from schema

Discussed in Chapter 4. Descriptions are free prompts. Skipping them is leaving 20-40% of accuracy on the table for ambiguous fields.

Bug 5: Using free-form strings where enums would work

// WRONG — model returns anything
"status": { "type": "string" }

// RIGHT — bounded set
"status": { 
  "type": "string", 
  "enum": ["draft", "review", "approved"] 
}

Bug 6: Not validating client-side "just because the provider validates"

Providers can have bugs. Schemas can be misinterpreted. Semantic checks (business rules) are never the provider's job. Always validate client-side.

// WRONG — trust the provider blindly
const rawInput = toolUse.input;
await saveToDatabase(rawInput);  // BUG if provider validation fails

// RIGHT — validate before persistence
const validated = PersonSchema.parse(toolUse.input);
await saveToDatabase(validated);

Bug 7: Reasoning fields placed after the answer

Discussed in Chapter 4 principle 10. Put reasoning BEFORE the answer field. If you put it after, the model computes the answer first (without thinking) and then rationalizes.

// WRONG — reasoning after answer
{
  "properties": {
    "sentiment": { "type": "string", "enum": ["positive", "neutral", "negative"] },
    "reasoning": { "type": "string" },
  }
}

// RIGHT — reasoning before answer
{
  "properties": {
    "reasoning": { "type": "string" },
    "sentiment": { "type": "string", "enum": ["positive", "neutral", "negative"] },
  }
}

Bug 8: Not handling truncated responses

Setting max_tokens too low for the schema. Response cuts off mid-object. JSON parsing fails. No sensible error message.

// WRONG — max_tokens too low
const response = await client.messages.create({
  max_tokens: 200,  // may cut off complex schemas
  tools: [{ name: "extract", input_schema: bigSchema }],
});

// RIGHT — generous max_tokens + truncation detection
const response = await client.messages.create({
  max_tokens: 2000,
  tools: [{ name: "extract", input_schema: bigSchema }],
});

if (response.stop_reason === "max_tokens") {
  metrics.increment("extraction.truncated");
  throw new Error("Response truncated; increase max_tokens");
}

Bug 9: Ignoring stop_reason

Beyond truncation, other stop_reasons indicate problems: refusal means the model refused; content_filter means content policy blocked. Silently failing on these produces bewildering user experiences.

if (response.stop_reason !== "tool_use" && 
    response.stop_reason !== "end_turn") {
  // Something unexpected happened
  metrics.increment("extraction.abnormal_stop", {
    reason: response.stop_reason,
  });
  throw new UnexpectedStopError(response.stop_reason);
}

Bug 10: Not versioning schemas

You update the schema; old cached responses no longer match. Or you regenerate old data with a new schema and get different results than the archive. Schemas need version numbers.

const PERSON_SCHEMA_VERSION = "v3";

const PersonSchemaV3 = z.object({
  _schema_version: z.literal("v3"),
  name: z.string(),
  age: z.number(),
  // v3 added:
  email: z.string().email().nullable(),
});


async function extractPerson(text: string) {
  const result = await extractWithSchema(text, PersonSchemaV3);
  return {
    ...result,
    _schema_version: PERSON_SCHEMA_VERSION,
  };
}


// Downstream code checks version
function normalizeToLatest(record: any): PersonV3 {
  if (record._schema_version === "v3") return record;
  if (record._schema_version === "v2") return migrateV2ToV3(record);
  throw new Error(`Unknown schema version: ${record._schema_version}`);
}

Bonus bug: composing structured output with tool use naively

When a workflow has both structured output extraction and tool-calling agent behavior, teams try to unify them into one schema and get tangled. Keep them separate: one call for extraction (structured output), one call for action (function calling). See G7 for the pattern.

The complete production-ready structured output helper

Everything from the previous chapters, composed into a helper you can drop into any TypeScript service. Provider-agnostic, schema-validated, repair-enabled, instrumented.

// structured-output.ts
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
import Anthropic from "@anthropic-ai/sdk";
import OpenAI from "openai";

// ---- Provider adapters ----

interface StructuredProvider {
  call<T>(
    text: string,
    schema: z.ZodType<T>,
    options?: { systemPrompt?: string; maxTokens?: number }
  ): Promise<{ parsed: T; raw: unknown }>;
}


class AnthropicStructuredProvider implements StructuredProvider {
  constructor(private client: Anthropic, private model = "claude-sonnet-4-6") {}
  
  async call<T>(text: string, schema: z.ZodType<T>, options: any = {}) {
    const jsonSchema = zodToJsonSchema(schema, "target") as any;
    const rootSchema = jsonSchema.definitions?.target ?? jsonSchema;
    
    const response = await this.client.messages.create({
      model: this.model,
      max_tokens: options.maxTokens ?? 2000,
      system: options.systemPrompt,
      tools: [{
        name: "record",
        description: "Record the extracted data",
        input_schema: rootSchema,
      }],
      tool_choice: { type: "tool", name: "record" },
      messages: [{ role: "user", content: text }],
    });
    
    if (response.stop_reason === "max_tokens") {
      throw new TruncatedResponseError();
    }
    
    const toolUse = response.content.find((b) => b.type === "tool_use") as any;
    if (!toolUse) throw new NoToolUseError();
    
    const parsed = schema.parse(toolUse.input);
    return { parsed, raw: toolUse.input };
  }
}


class OpenAIStructuredProvider implements StructuredProvider {
  constructor(private client: OpenAI, private model = "gpt-5.6") {}
  
  async call<T>(text: string, schema: z.ZodType<T>, options: any = {}) {
    const jsonSchema = zodToJsonSchema(schema, "target") as any;
    const rootSchema = jsonSchema.definitions?.target ?? jsonSchema;
    
    const response = await this.client.chat.completions.create({
      model: this.model,
      max_tokens: options.maxTokens ?? 2000,
      messages: [
        ...(options.systemPrompt 
          ? [{ role: "system" as const, content: options.systemPrompt }]
          : []),
        { role: "user", content: text },
      ],
      response_format: {
        type: "json_schema",
        json_schema: {
          name: "extraction",
          strict: true,
          schema: rootSchema,
        },
      },
    });
    
    if (response.choices[0].finish_reason === "length") {
      throw new TruncatedResponseError();
    }
    
    const content = response.choices[0].message.content;
    if (!content) throw new NoContentError();
    
    const rawObj = JSON.parse(content);
    const parsed = schema.parse(rawObj);
    return { parsed, raw: rawObj };
  }
}


// ---- Error types ----

class TruncatedResponseError extends Error {
  constructor() { super("Response truncated; increase max_tokens"); }
}
class NoToolUseError extends Error {
  constructor() { super("Model did not call the extraction tool"); }
}
class NoContentError extends Error {
  constructor() { super("Model returned no content"); }
}


// ---- Metrics hooks ----

interface Metrics {
  onAttempt(schemaName: string): void;
  onSuccess(schemaName: string, attempts: number): void;
  onFailure(schemaName: string, field: string, code: string): void;
  onRepair(schemaName: string): void;
  onGiveUp(schemaName: string, reason: string): void;
}

const defaultMetrics: Metrics = {
  onAttempt: () => {},
  onSuccess: () => {},
  onFailure: () => {},
  onRepair: () => {},
  onGiveUp: () => {},
};


// ---- The extraction function ----

interface ExtractOptions<T, F> {
  schema: z.ZodType<T>;
  fallbackSchema?: z.ZodType<F>;
  schemaName: string;
  systemPrompt?: string;
  maxTokens?: number;
  maxRepairAttempts?: number;
  metrics?: Metrics;
}


export async function extract<T, F = T>(
  provider: StructuredProvider,
  text: string,
  options: ExtractOptions<T, F>,
): Promise<{ value: T | F; usedFallback: boolean; attempts: number }> {
  const {
    schema, fallbackSchema, schemaName,
    systemPrompt, maxTokens,
    maxRepairAttempts = 2,
    metrics = defaultMetrics,
  } = options;
  
  metrics.onAttempt(schemaName);
  
  // Step 1: Try direct extraction
  try {
    const { parsed } = await provider.call(text, schema, {
      systemPrompt, maxTokens,
    });
    metrics.onSuccess(schemaName, 1);
    return { value: parsed, usedFallback: false, attempts: 1 };
  } catch (e) {
    recordFailure(metrics, schemaName, e);
    if (e instanceof TruncatedResponseError) throw e;  // don't retry
  }
  
  // Step 2: Re-ask with error feedback
  let lastError: unknown;
  for (let attempt = 0; attempt < maxRepairAttempts; attempt++) {
    metrics.onRepair(schemaName);
    try {
      const errorContext = lastError instanceof z.ZodError
        ? `Previous errors: ${lastError.issues.map(
            (i) => `${i.path.join(".")}: ${i.message}`).join("; ")}`
        : "";
      
      const { parsed } = await provider.call(
        text,
        schema,
        {
          systemPrompt: `${systemPrompt ?? ""}\n${errorContext}`,
          maxTokens,
        }
      );
      metrics.onSuccess(schemaName, attempt + 2);
      return { value: parsed, usedFallback: false, attempts: attempt + 2 };
    } catch (e) {
      recordFailure(metrics, schemaName, e);
      lastError = e;
    }
  }
  
  // Step 3: Fall back to looser schema if provided
  if (fallbackSchema) {
    try {
      const { parsed } = await provider.call(text, fallbackSchema, {
        systemPrompt, maxTokens,
      });
      metrics.onSuccess(schemaName + ".fallback", 1);
      return { 
        value: parsed as any, 
        usedFallback: true, 
        attempts: maxRepairAttempts + 2 
      };
    } catch (e) {
      recordFailure(metrics, schemaName + ".fallback", e);
    }
  }
  
  metrics.onGiveUp(schemaName, "exhausted");
  throw lastError ?? new Error("Extraction failed");
}


function recordFailure(metrics: Metrics, schemaName: string, e: unknown) {
  if (e instanceof z.ZodError) {
    for (const issue of e.issues) {
      metrics.onFailure(schemaName, issue.path.join("."), issue.code);
    }
  } else {
    metrics.onFailure(schemaName, "_root", "non_zod");
  }
}

Using it in production

import { Counter } from "prom-client";

// Prometheus metrics
const attemptCounter = new Counter({
  name: "extraction_attempts_total",
  help: "Extraction attempts",
  labelNames: ["schema"],
});
const successCounter = new Counter({
  name: "extraction_success_total",
  help: "Successful extractions",
  labelNames: ["schema"],
});
const failureCounter = new Counter({
  name: "extraction_failure_total",
  help: "Field-level failures",
  labelNames: ["schema", "field", "code"],
});
const repairCounter = new Counter({
  name: "extraction_repair_total",
  help: "Repair attempts",
  labelNames: ["schema"],
});
const giveUpCounter = new Counter({
  name: "extraction_giveup_total",
  help: "Give-ups",
  labelNames: ["schema", "reason"],
});


const metrics = {
  onAttempt: (s: string) => attemptCounter.labels(s).inc(),
  onSuccess: (s: string) => successCounter.labels(s).inc(),
  onFailure: (s: string, field: string, code: string) =>
    failureCounter.labels(s, field, code).inc(),
  onRepair: (s: string) => repairCounter.labels(s).inc(),
  onGiveUp: (s: string, r: string) => giveUpCounter.labels(s, r).inc(),
};


// Define schemas
const PersonSchema = z.object({
  name: z.string().min(1),
  age: z.number().int().min(0).max(150),
  email: z.string().email().nullable(),
  role: z.enum(["engineer", "manager", "designer", "other"]),
  reasoning: z.string().describe(
    "Brief note on how you determined the role from the text"
  ),
});
type Person = z.infer<typeof PersonSchema>;


const PersonLooseSchema = z.object({
  name: z.string().min(1),
  age: z.number().int().nullable(),
  email: z.string().nullable(),
  role: z.string().nullable(),
});


// Set up provider
const anthropic = new Anthropic();
const provider = new AnthropicStructuredProvider(anthropic);


// Extract with full pipeline
async function extractPerson(text: string): Promise<Person> {
  const { value, usedFallback, attempts } = await extract(provider, text, {
    schema: PersonSchema,
    fallbackSchema: PersonLooseSchema,
    schemaName: "person",
    systemPrompt: "Extract person details. Use 'other' for unclear roles.",
    maxTokens: 1000,
    maxRepairAttempts: 2,
    metrics,
  });
  
  if (usedFallback) {
    await queueForReview({ text, result: value }, "loose_schema_used");
  }
  
  console.log(`Extracted after ${attempts} attempts`);
  return value as Person;
}

What this gives you

• Provider-agnostic (Anthropic and OpenAI adapters, easy to add Gemini)
• Schema-enforced generation via provider primitives
• Client-side Zod validation for constraint checking
• Automatic repair with error feedback (2 attempts by default)
• Graceful degradation to looser schema when strict fails
• Full metrics on attempts, successes, field-level failures, repairs, give-ups
• Reasoning field pattern for improved accuracy on ambiguous fields
• Truncation detection (fails fast; don't retry infinitely)

What it doesn't do

• Streaming — separate helper. See Chapter 9 patterns.
• Multi-step pipelines — compose calls to this helper. See Chapter 8.
• Cross-record validation — your business logic layer.
• Grounding verification — add quote-alongside pattern to your schema (see Chapter 6).

Composing with other reliability patterns

This helper sits inside the reliability quartet's request layer. Compose with:

• Exponential backoff (G9) — wrap the extraction call in backoff for provider transient errors
• Circuit breakers (G3) — short-circuit during sustained provider outages
• Fallback strategies (G4) — provider adapter switches at the boundary of this helper
• Function calling reliability (G7) — the same patterns for when the extraction IS an action, not just data

Frequently asked questions

Library & further reading

Everywhere else on AI Error Hub that touches structured output, schemas, and adjacent reliability patterns.