Structured Output & JSON Mode: Get Reliable AI Responses Across Providers

Structured Output & JSON Mode: Get Reliable AI Responses Across Providers#

One of the biggest challenges in building AI-powered applications is getting models to return data in a consistent, parseable format. Structured output and JSON mode solve this by constraining model responses to valid JSON that matches your schema.

This guide compares how different AI providers implement structured output and shows you how to use each one effectively.

What is Structured Output?#

Structured output forces an AI model to return responses in a specific format — typically JSON conforming to a JSON Schema. Instead of hoping the model returns valid JSON, the API guarantees it.

Why It Matters#

code

Without structured output:
  User: "Extract the name and email from this text"
  AI: "The name is John Smith and the email is john@example.com"
  → You need regex/parsing to extract data 😩

With structured output:
  AI: {"name": "John Smith", "email": "john@example.com"}
  → Direct JSON.parse(), ready to use ✅

Key Benefits#

100% valid JSON — no parsing failures in production
Schema enforcement — responses match your exact data model
Type safety — numbers are numbers, arrays are arrays
Reduced hallucination — constrained output space limits fabrication

Provider Comparison#

Feature	OpenAI	Anthropic (Claude)	Google (Gemini)	DeepSeek
JSON Mode	✅ `response_format: {type: "json_object"}`	✅ Via system prompt	✅ `response_mime_type`	✅ `response_format`
Schema Enforcement	✅ `json_schema` type	⚠️ Prompt-based	✅ `response_schema`	⚠️ Prompt-based
Strict Mode	✅ `strict: true`	❌	✅	❌
Tool-based	✅ Function calling	✅ Tool use	✅ Function calling	✅ Function calling
Streaming	✅ Partial JSON	✅	✅	✅
Nested Objects	✅	✅	✅	✅
Arrays	✅	✅	✅	✅
Enums	✅	✅	✅	⚠️ Limited

Implementation Guide#

OpenAI — Structured Outputs (Recommended)#

python

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    api_key="YOUR_CRAZYROUTER_KEY",
    base_url="https://crazyrouter.com/v1"
)

# Define your schema with Pydantic
class ProductReview(BaseModel):
    product_name: str
    rating: float
    pros: list[str]
    cons: list[str]
    summary: str
    recommended: bool

# Use structured output - guaranteed to match schema
response = client.beta.chat.completions.parse(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Extract a structured product review from the text."},
        {"role": "user", "content": "I bought the Sony WH-1000XM6 headphones last month. Amazing noise cancellation, super comfortable for long flights. Battery lasts 40 hours which is insane. The app is a bit clunky though, and they're pricey at $399. Sound quality is phenomenal, especially for classical music. Highly recommend if you can afford them. 9/10."}
    ],
    response_format=ProductReview
)

review = response.choices[0].message.parsed
print(f"Product: {review.product_name}")
print(f"Rating: {review.rating}/10")
print(f"Recommended: {review.recommended}")
print(f"Pros: {', '.join(review.pros)}")

OpenAI — JSON Schema (Raw)#

python

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Extract events from the text as JSON."},
        {"role": "user", "content": "The conference is March 15-17 in San Francisco. Keynote by Sam Altman on March 15 at 9 AM. Workshop on fine-tuning March 16 at 2 PM."}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "events",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "events": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "title": {"type": "string"},
                                "date": {"type": "string"},
                                "time": {"type": "string"},
                                "location": {"type": "string"}
                            },
                            "required": ["title", "date"],
                            "additionalProperties": False
                        }
                    }
                },
                "required": ["events"],
                "additionalProperties": False
            }
        }
    }
)

import json
events = json.loads(response.choices[0].message.content)

Claude — Tool-Based Structured Output#

python

import anthropic

client = anthropic.Anthropic(
    api_key="YOUR_CRAZYROUTER_KEY",
    base_url="https://crazyrouter.com/v1"
)

response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    tools=[{
        "name": "extract_review",
        "description": "Extract a structured product review",
        "input_schema": {
            "type": "object",
            "properties": {
                "product_name": {"type": "string"},
                "rating": {"type": "number", "minimum": 0, "maximum": 10},
                "pros": {"type": "array", "items": {"type": "string"}},
                "cons": {"type": "array", "items": {"type": "string"}},
                "summary": {"type": "string"},
                "recommended": {"type": "boolean"}
            },
            "required": ["product_name", "rating", "pros", "cons", "summary", "recommended"]
        }
    }],
    tool_choice={"type": "tool", "name": "extract_review"},
    messages=[{
        "role": "user",
        "content": "Extract a review from: Great laptop, the M4 chip is blazing fast. Battery life is incredible at 18 hours. A bit heavy at 4.7 lbs. The keyboard could be better. 8.5/10, recommended for developers."
    }]
)

# The response is guaranteed valid JSON matching the schema
review_data = response.content[0].input
print(review_data)

Gemini — Response Schema#

python

import google.generativeai as genai

genai.configure(api_key="YOUR_API_KEY")

model = genai.GenerativeModel("gemini-2.5-flash")

response = model.generate_content(
    "Extract the recipe from: Mix 2 cups flour, 1 cup sugar, 3 eggs. Bake at 350°F for 30 minutes.",
    generation_config=genai.GenerationConfig(
        response_mime_type="application/json",
        response_schema={
            "type": "object",
            "properties": {
                "recipe_name": {"type": "string"},
                "ingredients": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "item": {"type": "string"},
                            "amount": {"type": "string"}
                        }
                    }
                },
                "instructions": {"type": "array", "items": {"type": "string"}},
                "cook_time_minutes": {"type": "integer"},
                "temperature_f": {"type": "integer"}
            }
        }
    )
)

import json
recipe = json.loads(response.text)

Universal Approach with Crazyrouter#

python

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_CRAZYROUTER_KEY",
    base_url="https://crazyrouter.com/v1"
)

# Works with ANY model through Crazyrouter
models = ["gpt-4o", "claude-sonnet-4-20250514", "gemini-2.5-flash"]

for model in models:
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": "Always respond in JSON format with keys: sentiment, confidence, keywords"},
            {"role": "user", "content": "Analyze: The new AI features are amazing but the pricing is too high"}
        ],
        response_format={"type": "json_object"}
    )
    print(f"{model}: {response.choices[0].message.content}")

Best Practices#

1. Always Validate Responses#

python

import json
from jsonschema import validate

schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "score": {"type": "number", "minimum": 0, "maximum": 100}
    },
    "required": ["name", "score"]
}

data = json.loads(response.choices[0].message.content)
validate(instance=data, schema=schema)  # Raises if invalid

2. Use Enums for Categorical Data#

python

response_format={
    "type": "json_schema",
    "json_schema": {
        "name": "classification",
        "schema": {
            "type": "object",
            "properties": {
                "category": {
                    "type": "string",
                    "enum": ["bug", "feature", "question", "documentation"]
                },
                "priority": {
                    "type": "string",
                    "enum": ["low", "medium", "high", "critical"]
                }
            }
        }
    }
}

3. Handle Streaming JSON#

python

# For streaming, collect chunks and parse at the end
full_content = ""
for chunk in client.chat.completions.create(
    model="gpt-4o",
    messages=[...],
    response_format={"type": "json_object"},
    stream=True
):
    if chunk.choices[0].delta.content:
        full_content += chunk.choices[0].delta.content

result = json.loads(full_content)

FAQ#

What's the difference between JSON mode and structured output?#

JSON mode guarantees the output is valid JSON but doesn't enforce a specific schema. Structured output (available in OpenAI and Gemini) additionally guarantees the JSON matches your exact schema with correct types, required fields, and constraints.

Which AI provider has the best structured output support?#

OpenAI currently leads with the most complete implementation including strict schema enforcement and Pydantic integration. Claude's tool-based approach is also reliable. Gemini's response_schema is effective but slightly less flexible. All are accessible through Crazyrouter with a unified API.

Does structured output increase latency?#

Slightly. Schema enforcement adds a small overhead (typically 50-200ms) due to constrained decoding. For most applications, this is negligible compared to the reliability gains.

Can I use structured output with streaming?#

Yes, all major providers support streaming with JSON mode. The model streams partial JSON tokens, and you concatenate them into a complete JSON object once the stream ends. OpenAI also supports partial JSON parsing during streaming.

How does structured output handle optional fields?#

In OpenAI's strict mode, all properties must be listed in required, and optional values should use "type": ["string", "null"] (nullable types). Claude's tool schema supports standard JSON Schema optional fields normally.

Summary#

Structured output is essential for building reliable AI-powered data pipelines. Whether you're extracting information, classifying content, or generating structured data, JSON mode and schema enforcement eliminate the fragility of free-text parsing.

With Crazyrouter, you can access structured output across all major providers — OpenAI, Claude, Gemini, DeepSeek, and more — through one unified API, with 25-30% cost savings.

Build reliable AI data pipelines → Get your Crazyrouter API key

Structured Output & JSON Mode: Get Reliable AI Responses Across Providers