Introduction

Azure OpenAI in 2026 lives inside Microsoft Foundry — the platform renamed from Azure AI Foundry (which itself was renamed from Azure AI Studio) on January 1, 2026. Same models as OpenAI direct — GPT-5.6 Sol/Terra/Luna, GPT-5.5, GPT-5.4, o-series — plus a growing catalog of non-OpenAI models including Claude (via a Hosted-on-Anthropic deployment), DeepSeek, Llama, and Grok. Different auth (Entra ID or API key), different deployment model (create resource → create deployment → call by deployment name), different pricing tiers (Global Standard, Data Zone, Regional), and PTU semantics that changed meaningfully in 2026.

This guide is the definitive reference for Azure OpenAI as it exists today. It covers Foundry's deployment model, Entra ID auth, PTU vs pay-as-you-go economics, content filters, On Your Data, and the enterprise infrastructure costs that inflate Azure bills by 15-40% vs OpenAI direct.

Who this guide is for

The guide assumes you have:

• An Azure subscription with Azure OpenAI access approved (application process required for new tenants)
• Made real Azure OpenAI API calls, whether via the openai Python SDK pointed at Azure, the azure-ai-openai SDK, or REST
• Hit at least one deployment-name-vs-model-ID confusion or 429 that sent you searching

If that's you, read this end-to-end once to build a full map of the API surface; return to specific chapters when you hit a related error. Each chapter links to the specific error-fix pages that answer narrower questions with tested code.

What Azure OpenAI is (and isn't)

Azure OpenAI is a proxy layer that serves OpenAI's models through Microsoft's infrastructure with Azure-native auth, networking, and compliance controls. Same GPT-5.6, GPT-5.5, GPT-5.4, o-series, GPT-4.1, and embeddings models as OpenAI direct. What you get on top: Entra ID authentication, VNet-integrated private endpoints, regional data residency, EA/MCA/MACC billing integration, PTU capacity reservations. What you don't get: the newest OpenAI features on day zero (Azure typically lags direct by 2-6 weeks), some newest capability integrations (Realtime API, some hosted tools).

The four things that changed in 2026

1. Rename to Microsoft Foundry (January 1, 2026). The platform was Azure AI Studio, then Azure AI Foundry, now Microsoft Foundry. URLs, docs, and CLI commands were updated. Old links may 404. If your team's runbooks still reference "AI Studio," they need updating. Same underlying platform, different name.

2. GPT-5.6 family live on Azure. Sol, Terra, and Luna are all deployable on Azure in Global and Data Zone flavors. Sol and Terra also have PTU deployments and Priority Processing tiers. If your Azure catalog knowledge predates July 2026, GPT-5.6 is the biggest update.

3. PTU reservations now work across the Foundry catalog. Previously, a PTU reservation was locked to a specific model family — buy for Azure OpenAI, only apply to OpenAI models. In 2026, reservations apply across supported Foundry models in the same region. A 500 PTU reservation bought for Azure OpenAI also covers DeepSeek-R1 or Llama 3.3 70B PTU deployments in the same region. This changes procurement math significantly.

4. Assistants API deprecation follows OpenAI direct. When OpenAI announced Assistants shutdown for August 26, 2026, Azure's earlier "we're unaffected" guidance changed. Azure OpenAI Assistants is now also scheduled for deprecation. Different exact timeline, same replacement path (Responses API + Conversations API). Chapter 5 covers this migration in the errors category.

What's not in this guide

• Deep coverage of general OpenAI API mechanics — see the OpenAI API pillar for Chat Completions, Responses API, structured outputs, and function calling. This guide focuses on what's different on Azure.
• Broader Azure AI (Speech, Vision, Language) — those are separate services with their own quirks
• Azure fundamentals (RBAC, VNet, private endpoints) — we assume you've deployed something on Azure before

API architecture at a glance

Azure OpenAI's API shape is similar to OpenAI direct but with three important differences: (1) requests go to your resource's endpoint URL, not api.openai.com; (2) you call by deployment name, not model ID; (3) every request needs an api-version query parameter. Understanding these differences is the single biggest source of first-day confusion on Azure OpenAI.

The Azure resource hierarchy

Azure OpenAI works on top of Azure's resource-group model. Every deployment lives inside a hierarchy:

1. Subscription — your Azure billing account
2. Resource group — a logical container within a subscription
3. Azure OpenAI resource — a specific instance with an endpoint URL (e.g. my-aoai-prod)
4. Deployment — a named deployment of a specific model within a resource (e.g. my-gpt-5-6-sol pointing to gpt-5.6-sol)

You call the deployment, not the model. This means the same underlying model can have multiple deployments with different quotas, different content filters, and different PTU vs pay-as-you-go configurations. It also means every Azure OpenAI call needs to know both the resource endpoint and the deployment name.

The endpoint URL pattern

POST https://{resource-name}.openai.azure.com/openai/deployments/{deployment-name}/chat/completions?api-version=2024-10-21
Authorization: Bearer <entra-id-token>
Content-Type: application/json

{
  "messages": [{"role": "user", "content": "Hello"}],
  "max_tokens": 100
}

Three unfamiliar pieces vs OpenAI direct:

• {resource-name}.openai.azure.com — your specific resource endpoint, not shared api.openai.com
• /openai/deployments/{deployment-name}/ — request routed by deployment name (which maps to model on Azure's side)
• ?api-version=2024-10-21 — API version pinning. Miss it or use a wrong one and you get 400 errors

The endpoint map

SDKs

from openai import AzureOpenAI

client = AzureOpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-10-21",
    azure_endpoint="https://my-aoai-prod.openai.azure.com/",
)

response = client.chat.completions.create(
    model="my-gpt-5-6-sol",   # ← deployment name, NOT model ID
    messages=[{"role": "user", "content": "Hello"}],
)

Request lifecycle

1. SDK serialization — Python/TypeScript objects turn into a JSON body
2. Auth — API key or Entra ID bearer token attached
3. Endpoint routing — request goes to your resource's regional endpoint
4. Deployment resolution — Azure maps deployment name to the actual model + capacity
5. Rate limit check — TPM/RPM enforced per deployment
6. Content filter pre-check — input evaluated against configured policy
7. Model inference — the underlying model generates the response
8. Content filter post-check — output evaluated; can block with a filter_result
9. Response streaming or buffering — response returned
10. Usage + billing — tokens counted, billed to your subscription

Errors fire at any stage. A 401 fires at step 2. A 404 (deployment not found) fires at step 4. A 429 fires at step 5. A content filter block returns 200 with a content_filter_result, not an error. Understanding which stage failed tells you where to look for the fix.

The core object model

Same core objects as OpenAI direct — Messages, Content Parts, Tools, Usage, Conversations — plus Azure-specific additions:

• content_filter_result — Azure adds per-response filter categories (hate, self_harm, sexual, violence, jailbreak, profanity) with severity thresholds
• prompt_filter_results — filter results for the input prompt (separate from output filter)
• Deployment name mapping — the model field in requests is your deployment name; the response model field is the actual model ID

Most errors are shape mismatches on these objects plus the deployment-name-vs-model-ID confusion. See the error categories chapter for the full taxonomy.

The 2026 Microsoft Foundry catalog

Microsoft Foundry hosts three categories of models: OpenAI models (the "Azure OpenAI" part), open-source and third-party models (DeepSeek, Llama, Grok, Mistral), and specialty models via partner deployments (Claude via Hosted-on-Anthropic, Sora for image generation). Each has different pricing, different regional availability, and different feature parity with the provider's direct API.

OpenAI models on Azure

Global Standard rates verified from azure.microsoft.com/pricing/details/azure-openai on August 19, 2026. Data Zone and Regional carry a rate premium for compliance-driven data residency. Priority Processing tiers on Sol and Terra add capacity guarantees at higher rates.

The three deployment types

Choose the loosest deployment type your compliance allows. Global Standard has the best availability and lowest cost. Regional Standard costs more and has less capacity elasticity — use only when required. See deployment type selection.

Non-OpenAI models on Foundry

Model selection: which one for what

On Azure OpenAI specifically:

• GPT-5.6 Luna ($0.20/$1.20) — classification, extraction, formatting
• GPT-5.6 Terra ($2/$12) — most production. Now cheaper than GPT-5.4 for equivalent quality
• GPT-5.6 Sol ($5/$30) — hardest reasoning, longest contexts
• GPT-4.1 ($2/$8) — legacy production workloads not yet migrated; still stable
• o4-mini ($1.10/$4.40) — reasoning workloads where you want the o-series behavior at lower cost
• Claude Sonnet 4.6 (via Hosted-on-Anthropic) — when you want Claude's reasoning inside your Azure ecosystem

For pure cost sensitivity at classification-tier workloads, DeepSeek-V3 on Foundry is often the cheapest option (well below Luna in some regions). Trade-off: less mature operational tooling than the OpenAI models on the same platform.

Sora and image generation

Sora — OpenAI's video-capable generative model — is now available in Azure AI Foundry with API-first integration. Deployed like other models via the Foundry portal, called via /images/generations or a Sora-specific endpoint depending on modality. Billed separately from token-based models. Regional availability limited to specific Azure regions.

What's coming and what's leaving

• GPT-4o and GPT-5 (base) removed from public model catalog on Azure — following OpenAI direct's removal
• Older GPT-3.5-turbo deployments still work but no new features
• Nova 2 preview available in select Azure regions (through Foundry partner deployment)
• Assistants API on Azure OpenAI following the direct-API deprecation timeline — migrate to Responses

Entra ID auth, resources & rate limits

Azure OpenAI supports two auth methods: API keys (like OpenAI direct) and Entra ID (Microsoft's identity platform, formerly Azure AD). Entra ID is the enterprise-recommended path — token-based, tied to Azure RBAC, works with managed identities for service-to-service calls. Rate limits are enforced per deployment as TPM and RPM, adjustable via quota-increase requests.

API key auth (simplest)

from openai import AzureOpenAI

client = AzureOpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-10-21",
    azure_endpoint="https://my-aoai-prod.openai.azure.com/",
)

API keys are simple to get started with but carry the same problems as user keys on OpenAI direct — they're bearer tokens that grant full access. Rotate regularly, store in Azure Key Vault, never commit to source. Get keys from the Foundry portal: Resource → Keys and Endpoint.

Entra ID auth (recommended for production)

Entra ID auth uses OAuth bearer tokens obtained from Microsoft's identity platform. Two patterns:

Managed identity (service-to-service)

from azure.identity import DefaultAzureCredential
from openai import AzureOpenAI

credential = DefaultAzureCredential()

def get_token():
    return credential.get_token("https://cognitiveservices.azure.com/.default").token

client = AzureOpenAI(
    azure_ad_token_provider=get_token,   # ← Entra ID auth
    api_version="2024-10-21",
    azure_endpoint="https://my-aoai-prod.openai.azure.com/",
)

DefaultAzureCredential tries multiple auth sources in order: environment variables, managed identity (if running on Azure), Azure CLI, VS Code, etc. In production on an Azure VM, App Service, Container Apps, or AKS, it picks up the managed identity automatically. No secret needed.

Client secret (service principal)

from azure.identity import ClientSecretCredential

credential = ClientSecretCredential(
    tenant_id=os.environ["AZURE_TENANT_ID"],
    client_id=os.environ["AZURE_CLIENT_ID"],
    client_secret=os.environ["AZURE_CLIENT_SECRET"],
)

Use when running outside Azure and can't attach a managed identity. Rotates via Entra ID app registration.

Required RBAC role

Entra ID auth requires the identity (managed identity, service principal, or user) to have the Cognitive Services OpenAI User role on the Azure OpenAI resource (or a scope that includes it). Missing this role returns 403 even with a valid token.

# Assign via Azure CLI
az role assignment create \
    --assignee <principal-id> \
    --role "Cognitive Services OpenAI User" \
    --scope "/subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.CognitiveServices/accounts/{resource-name}"

Additional roles: Cognitive Services OpenAI Contributor for management operations (creating deployments, etc.); Cognitive Services Contributor for full control. See RBAC role assignment.

Rate limits & TPM quotas

Azure enforces rate limits per deployment, not per model or account. Each deployment gets a TPM quota when created, split from your subscription's model-level quota. The RPM limit is derived from TPM at a fixed ratio (approximately 6 RPM per 1,000 TPM for most models).

Subscription-level quota is your total TPM budget for a model + region combination. Deployments carve from this budget. If your subscription has 300K TPM of GPT-5.6 Sol in East US, you can create three deployments at 100K TPM each, or one at 300K, or any mix.

Requesting quota increase

Default quotas are conservative and rarely enough for production. Request increases via the Azure portal:

1. Portal → Azure OpenAI resource → Quotas in the left nav
2. Filter by model + region
3. Click Request quota → fill the form with justification
4. Approval typically 24-72 hours; larger requests may need manual review

See quota increase request.

Dynamic quota vs fixed TPM

A 2026 Azure feature: Dynamic quota lets Standard-tier deployments burst above their fixed TPM allocation when regional capacity is available. Enabled per deployment. Requests above the fixed TPM get served at Standard rates (not the 429'd baseline). During capacity crunches, dynamic quota reverts to fixed. See dynamic quota.

Common auth & quota errors

• 401 Unauthorized — API key wrong or Entra ID token expired/invalid. Verify token scope and expiration
• 403 Permission denied — role assignment missing, or scope on token doesn't cover this resource
• 404 Deployment not found — deployment name typo, or deployment created in a different resource
• 429 Too Many Requests — TPM or RPM quota hit. Implement backoff; consider quota increase
• 503 Service Unavailable — Azure-side capacity issue. Retry with backoff

Every Azure OpenAI error category

Azure OpenAI errors span two worlds: Azure platform errors (auth, quotas, resource management) and the underlying model errors (context length, validation, content filters). This chapter maps ten categories and links to specific error-fix pages.

1. Resource & deployment errors

Fires at: Request routing, before hitting a model.
Common causes: Deployment name wrong or doesn't exist; deployment created in a different resource than you're calling; model not deployed at all (you deployed a resource but not a model within it); wrong region.

The single most common first-day error on Azure OpenAI is passing a model ID (like gpt-5.6-sol) where the deployment name is expected. Azure returns 404 with "The API deployment for this resource does not exist" — the fix is passing the deployment name you created in the Foundry portal (e.g. my-gpt-5-6-sol-prod).

Related pages: Deployment name vs model ID · Model not in region

2. Authentication errors (401, 403)

Fires at: Auth stage, before request processing.
Common causes: Invalid API key; expired Entra ID token; missing RBAC role assignment; wrong scope on token; managed identity not assigned to the caller resource.

On Entra ID auth specifically, verify: (1) the identity has the Cognitive Services OpenAI User role, (2) the token was acquired for the correct scope, (3) the token hasn't expired (default lifetime is 60-90 minutes). DefaultAzureCredential handles refresh; static tokens don't.

Related pages: 401 invalid API key · Entra ID token · Scope mismatch

3. API version errors

Fires at: Request parsing.
Common causes: Missing api-version query parameter; using a preview api-version that no longer exists; using a GA api-version older than what a feature requires.

Azure OpenAI's api-version pinning is essential — different versions expose different features. 2024-10-21 is a current GA version; 2024-preview gets you newer features but with less stability. Missing the parameter entirely returns 400 with a message about the API version being required.

Related pages: api-version mismatch · preview vs GA

4. Rate limit errors (429)

Fires at: Rate limit layer, before inference.
Common causes: TPM quota hit for a deployment; RPM quota hit; subscription-level quota exhausted across deployments; PTU capacity saturated.

Same fix as OpenAI direct 429s: exponential backoff with jitter, respect retry-after header. Additionally on Azure: consider dynamic quota to burst above fixed TPM, or request a quota increase for the deployment. On PTU deployments, throttling means you've saturated your reserved capacity — either add PTU or accept the ceiling.

Related pages: 429 TPM quota · Quota increase

5. Request shape errors (400)

Fires at: Request validation, before inference.
Common causes: Same as OpenAI direct — invalid role sequence, missing required fields, unsupported parameter for the model, malformed content. Plus Azure-specific: content filter policy violations detected at request time.

Since Azure OpenAI proxies to the same underlying models as OpenAI direct, most 400s are identical to what you'd see on api.openai.com. Reference the OpenAI API pillar for the base error taxonomy; the Azure-specific 400s are described in the deployment/api-version categories above.

6. Content filter errors (returned as 200 with filter_result)

Fires at: Content filter pre-check or post-check.
Common causes: Input triggers hate/self_harm/sexual/violence/jailbreak/profanity thresholds; output triggers same categories; jailbreak detection flagged the prompt as a bypass attempt.

Content filter results come back inside a 200 response, not as an error. Check prompt_filter_results (input) and choices[0].content_filter_results (output) for filter categories. If severity exceeds threshold, the model may return truncated or empty content. Chapter 8 covers content filters in depth.

Related pages: Content filter triggered · Jailbreak detection

7. PTU-specific errors

Fires at: PTU deployment creation or invocation.
Common causes: Insufficient PTU quota; below minimum PTU allocation (15 for Global/Data Zone, 50 for Regional); PTU capacity not available in the requested region; utilization above reserved capacity.

PTU has its own set of errors distinct from pay-as-you-go. Getting quota is a two-step process (request PTU quota, then create deployment). Chapter 7 walks the PTU model.

Related pages: PTU minimum · PTU capacity not available

8. On Your Data & Azure AI Search errors

Fires at: RAG retrieval or grounding stage.
Common causes: Search index permissions not granted; index doesn't exist; vector configuration missing; embedding model deployment not accessible from OpenAI resource.

On Your Data integrates Azure AI Search into Chat Completions for RAG. Failures span both services — index config on the Search side, permissions on both sides, embedding model deployment on the OpenAI side. Chapter 9 covers the integration.

Related pages: Search index permissions · Vector index config

9. Assistants API deprecation errors

Fires at: Any Assistants endpoint after the Azure deprecation deadline.
Common causes: Code still using /assistants or /threads endpoints; Azure earlier said "unaffected" — that guidance changed in 2026.

When OpenAI direct announced Assistants shutdown for August 26, 2026, Azure OpenAI's earlier "we're unaffected" guidance was reversed. Azure OpenAI Assistants is now on a deprecation timeline. Verify the current shutdown date in Azure docs; plan migration to Responses API on the same timeline as OpenAI direct.

Related pages: Assistants deprecation on Azure · Migration path

10. Network & private endpoint errors

Fires at: Network layer, before AI processing.
Common causes: Private endpoint not configured; VNet peering missing; NSG rules blocking traffic; DNS resolution not pointed at private endpoint IP.

Enterprise deployments use private endpoints to keep traffic off the public internet. Misconfiguration returns connection timeouts rather than clean HTTP errors, making debugging harder. Common trap: creating a private endpoint but leaving DNS resolving to the public endpoint. See private endpoint VNet config.

Deployment names, types & endpoint URLs

The single biggest conceptual difference between Azure OpenAI and OpenAI direct is the deployment layer. You don't call models by name — you create a named deployment of a model, then call the deployment. Every request goes to a deployment-specific URL. Understanding this layer is essential for everything else in Azure OpenAI.

The lifecycle of a deployment

1. Get subscription quota for the model + region combination (default quota exists but often needs increase)
2. Create an Azure OpenAI resource in a chosen region (this gives you an endpoint URL)
3. Create a deployment inside that resource: pick model, deployment type (Global/Data Zone/Regional/PTU), TPM allocation, content filter, deployment name
4. Configure content filter for the deployment (or use default)
5. Call the deployment by its name via the resource endpoint URL

The deployment name is your choice — it's just a label. Common patterns: <model>-<env> (e.g. gpt-5-6-sol-prod), <team>-<model> (e.g. support-gpt-5-6-terra), or purpose-based (chat-flagship, classifier). Pick a convention that survives model swaps — a deployment named chat-flagship can transparently swap from GPT-5.5 to GPT-5.6 Sol without app code changes.

Deployment types compared

See deployment type selection.

The endpoint URL anatomy

Every Azure OpenAI call goes to a URL that encodes the resource, deployment, and API version:

https://{resource-name}.openai.azure.com/openai/deployments/{deployment-name}/{operation}?api-version={version}

# Examples
https://my-aoai-prod.openai.azure.com/openai/deployments/gpt-5-6-sol-prod/chat/completions?api-version=2024-10-21
https://my-aoai-prod.openai.azure.com/openai/deployments/gpt-5-6-sol-prod/responses?api-version=2024-preview
https://my-aoai-prod.openai.azure.com/openai/deployments/text-embed-large/embeddings?api-version=2024-10-21

The {operation} portion (chat/completions, responses, embeddings, images/generations, etc.) is where the API behavior differs. The {resource-name} and {deployment-name} route to your specific deployment. The api-version selects the API contract shape.

api-version pinning

Every request must include an api-version query parameter. Miss it and Azure returns 400 with "The API version parameter is required." Wrong or expired preview versions return 400 with "The API version is not valid."

Common versions in 2026:

• 2024-10-21 — recent GA, broad compatibility
• 2024-preview or 2024-preview-* — preview versions with newer features (Responses API, latest structured outputs)
• 2024-08-01-preview — Assistants API v2
• 2024-05-01-preview — earlier preview with GPT-4o support

Preview versions may be superseded or retired; check Azure docs for currently supported preview versions in your region. See preview vs GA.

Multiple deployments of the same model

Nothing prevents creating multiple deployments of the same model within one resource:

• gpt-5-6-sol-prod — Global Standard, 100K TPM, strict content filter
• gpt-5-6-sol-dev — Global Standard, 20K TPM, relaxed content filter
• gpt-5-6-sol-ptu — Global PTU, 30 PTU reserved
• gpt-5-6-sol-support — Data Zone Standard, own quota for support team

This pattern is powerful for teams sharing an Azure OpenAI resource — each team gets its own deployment with its own quota, content filter, and cost attribution. It's also how you separate PTU-reserved workloads from pay-as-you-go workloads on the same model.

PTU, reservations & the pricing math

Provisioned Throughput Units are Azure's reserved-capacity model — you pay an hourly rate per PTU whether you use the capacity or not, in exchange for guaranteed throughput and predictable per-token cost. In 2026, PTU reservations gained cross-model portability: buy once, apply across the Foundry catalog in the same region. This chapter walks the economics, the minimums, and when PTU actually wins.

PTU minimums by deployment type

• Global PTU: 15 PTU minimum
• Data Zone PTU: 15 PTU minimum
• Regional PTU: 50 PTU minimum for most models, 25 PTU for mini models (5.6 Luna, 5.4-mini)

Below the minimum, you can't create a PTU deployment at all. This makes Regional PTU particularly expensive to get started with — 50 PTU × hourly rate is a real commit even if you use only 10 PTU worth of capacity. See PTU minimums.

PTU pricing math

PTU rates vary by model. As a rough starting point in 2026:

• GPT-5.6 Sol PTU: roughly $40-$50/hr per PTU (depends on region and reservation)
• GPT-5.6 Terra PTU: roughly $18-$25/hr per PTU
• GPT-5.6 Luna PTU: roughly $6-$10/hr per PTU (mini tier)

Break-even example for GPT-5.6 Sol at 15 PTU (Global minimum):

• 15 PTU × $45/hr × 24 × 30 = $486,000/month (upper bound estimate)
• 15 PTU typically supports roughly 50-100K TPM sustained (varies by workload shape)
• Equivalent pay-as-you-go: 50-100K TPM × 60 × 24 × 30 × $5/1M input = $10.8-$21.6K/month input, plus output

These numbers show PTU is only economical at very high, very predictable utilization. Underutilized PT is a very expensive way to pay for idle capacity. Rule of thumb: PT breaks even around 60-70% sustained utilization vs equivalent Standard pay-as-you-go.

PTU reservations — the 2026 game-changer

Reservations let you commit to a monthly or annual PTU allocation for a discount vs on-demand hourly PTU. In 2026, a major change landed: reservations work across the Foundry Models catalog. A 500 PTU reservation bought for Azure OpenAI now applies to any supported Foundry model deployed as PTU in the same region and scope.

Implication: procurement math changes. Forecast your total PTU need across models, buy one reservation that covers your steady state, deploy across models as needed. Pay hourly only for spike capacity above the reservation. See cross-model reservations.

PTU vs Standard — decision matrix

PTU-specific errors

• Insufficient PTU quota — request via quota form; larger requests need manual review
• PTU capacity not available in region — try different region or wait; Azure has finite hardware
• InvalidResourceProperties on deployment creation — usually below-minimum PTU count
• 429 on PTU deployment — you've saturated your reserved capacity; add PTU or accept throttling
• Underutilization — no error, just billing surprise. Monitor and adjust

Content filters & Responsible AI

Every Azure OpenAI request runs through content filters — pre-check on input, post-check on output. Filters cover six categories (hate, self_harm, sexual, violence, jailbreak, profanity) with four severity levels each. When a filter fires, the response comes back as a 200 with content_filter_result, not an error. Handling filter results correctly is essential for production apps.

The six filter categories

Each category has four severity thresholds: safe, low, medium, high. You configure the threshold per deployment — content at or above the threshold gets filtered. See filter thresholds.

Reading filter results

Filter results come back inside a 200 response:

response = client.chat.completions.create(
    model="my-gpt-5-6-sol-prod",
    messages=[{"role": "user", "content": user_input}],
)

# Prompt filter results (input evaluation)
if hasattr(response, "prompt_filter_results"):
    for pf in response.prompt_filter_results:
        cfr = pf["content_filter_results"]
        if cfr.get("hate", {}).get("filtered"):
            handle_input_blocked("hate")
        if cfr.get("jailbreak", {}).get("detected"):
            handle_jailbreak_attempt()

# Response filter results (output evaluation)
choice = response.choices[0]
if choice.finish_reason == "content_filter":
    cfr = choice.content_filter_results
    for category in ["hate", "self_harm", "sexual", "violence"]:
        if cfr.get(category, {}).get("filtered"):
            handle_output_blocked(category, cfr[category]["severity"])

Two signals: filtered: true means content was blocked; severity tells you why (low/medium/high). See filter triggered handling.

Jailbreak detection

The jailbreak category is specifically for prompt injection detection — attempts to bypass system prompts or safety filters. Unlike the other categories, jailbreak has just a boolean detected field, no severity levels. Detected jailbreak attempts should almost always be logged (for security monitoring) and returned to the user with a generic decline message. Do NOT return the raw jailbreak attempt to any other user via a shared session.

See jailbreak detection.

Configuring custom filter policies

Default filter policies are moderately strict. For legitimate use cases where the defaults over-block (medical content flagged as self_harm; adult platforms needing sexual content), you can:

1. Adjust per-category thresholds — set specific categories to high only (less strict) or leave at medium
2. Create a custom content filter policy in the Foundry portal with your thresholds
3. Attach the custom policy to specific deployments — different deployments can have different filters
4. Request Modified Content Filtering for approved use cases requiring even lower thresholds — a formal application to Microsoft with justification

Abuse monitoring & opt-out

By default, Azure OpenAI performs abuse monitoring: logs prompts and completions for up to 30 days, reviews flagged content for policy violations. For customers with strict data-handling requirements (healthcare, legal), you can apply to opt out of abuse monitoring. Approval requires justification and typically a Microsoft enterprise relationship. See abuse monitoring opt-out.

Content filter costs

Content filters are included in the model pricing — no separate line item. This is a notable difference from Bedrock Guardrails, which are billed separately per evaluation. If you're comparing total cost of Azure OpenAI vs Bedrock for the same workload, factor in that Azure includes filtering while Bedrock adds it as a cost.

Common content filter pitfalls

• Not checking filter results — assuming a 200 always has usable content
• Not distinguishing prompt filters from response filters in error handling
• Treating jailbreak detection as low-priority — it's a security signal, log it
• Not testing with your actual user content — defaults may over-filter your legitimate use case
• Applying strict filters in dev but lax in prod (or vice versa) — inconsistent behavior between environments
• Never using Modified Content Filtering when your use case legitimately requires it

On Your Data & Azure AI Search

On Your Data is Azure OpenAI's built-in RAG integration — attach an Azure AI Search index to a Chat Completions call, get grounded responses with citations. Powerful when configured correctly, opaque when it fails. This chapter walks the setup, the permission model that trips up most teams, and the citation format.

What On Your Data does

A standard Chat Completions call generates a response from the model's parameters. On Your Data adds a step: before generation, Azure OpenAI queries an Azure AI Search index (or another supported data source), pulls relevant chunks, and injects them into the prompt. The model generates from your data plus its parameters, and the response includes citations pointing back to the source documents.

Supported data sources: Azure AI Search, Azure Cosmos DB for MongoDB vCore, Azure Blob Storage (for basic file grounding), Elasticsearch, Pinecone, MongoDB Atlas.

The minimum working setup

from openai import AzureOpenAI

client = AzureOpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-10-21",
    azure_endpoint="https://my-aoai-prod.openai.azure.com/",
)

response = client.chat.completions.create(
    model="my-gpt-5-6-sol-prod",
    messages=[{"role": "user", "content": "What's our refund policy?"}],
    extra_body={
        "data_sources": [
            {
                "type": "azure_search",
                "parameters": {
                    "endpoint": "https://my-search.search.windows.net",
                    "index_name": "policies-index",
                    "authentication": {
                        "type": "system_assigned_managed_identity",
                    },
                    "query_type": "vector_semantic_hybrid",
                    "embedding_dependency": {
                        "type": "deployment_name",
                        "deployment_name": "text-embed-large",
                    },
                    "fields_mapping": {
                        "content_fields": ["content"],
                        "title_field": "title",
                        "url_field": "url",
                        "filepath_field": "filepath",
                    },
                    "in_scope": True,
                    "role_information": "You are a helpful assistant answering from company policies.",
                    "strictness": 3,
                    "top_n_documents": 5,
                },
            }
        ],
    },
)

# Response includes citations
message = response.choices[0].message
print(message.content)   # Answer text with citation markers [doc1], [doc2]

# Access citations
for citation in message.context.get("citations", []):
    print(f"[{citation['title']}]({citation['url']})")

The permission chain — where most teams get stuck

On Your Data requires permissions across three services: your Azure OpenAI resource, your Azure AI Search resource, and (if used) your embedding model deployment. The chain:

1. Your identity (managed identity or user) needs Cognitive Services OpenAI User on the OpenAI resource
2. Azure OpenAI's system-assigned managed identity needs Search Index Data Reader on the Search index
3. Azure OpenAI's managed identity also needs Search Service Contributor on the Search service (for some index operations)
4. If using vector search: the embedding model deployment must be accessible from the OpenAI resource

Miss any one of these and On Your Data fails silently (returns a response as if there was no data source) or with an obscure 400 pointing to the search integration. See search index permissions.

query_type — the retrieval mode

The query_type parameter controls how Search finds relevant chunks:

• simple — keyword-only search, no vectors
• semantic — Azure AI Search semantic ranking (LLM re-rank of keyword results)
• vector — pure vector similarity, requires embedding_dependency configured
• vector_simple_hybrid — keyword + vector hybrid
• vector_semantic_hybrid — keyword + vector + semantic re-rank (best quality, highest cost)

For most workloads, vector_semantic_hybrid gives the best results but requires a semantic-ranker-enabled Search tier (S1 or higher) and a deployed embedding model. See vector search config.

Citation parsing

On Your Data returns citations in the response's message.context field:

{
  "content": "Our refund policy allows 30 days [doc1] with proof of purchase [doc2].",
  "context": {
    "citations": [
      {
        "content": "Full text of the source chunk...",
        "title": "Refund Policy 2026",
        "url": "https://intranet/policies/refunds-2026.pdf",
        "filepath": "policies/refunds-2026.pdf",
        "chunk_id": "0"
      },
      ...
    ],
    "intent": "..." // What the model interpreted the user query as
  }
}

Match citation markers in content ([doc1], [doc2]) to citations by 1-indexed position. Display these as clickable references in your UI. See citation parsing.

strictness parameter

The strictness parameter (1-5) controls how closely the model must ground its answer in retrieved chunks:

• 1-2 — model can freely mix retrieved data with parametric knowledge
• 3 — balanced (default)
• 4-5 — model refuses if not clearly supported by retrieved chunks

Higher strictness reduces hallucination but increases refusal rate for questions your index doesn't cover well. Tune based on user tolerance for "I don't know" responses.

Pricing & the enterprise cost picture

Azure OpenAI's model pricing tracks OpenAI direct closely on Global Standard — within 5% for most models. But Azure's total cost is 15-40% higher in enterprise deployments because of the supporting infrastructure the pricing page doesn't show. This chapter walks the visible pricing, the enterprise cost stack, and where the actual money goes.

Model pricing — Global Standard tier

Rates verified from azure.microsoft.com/pricing/details/azure-openai on August 19, 2026. Data Zone Standard adds ~5-10% premium. Regional Standard adds more. Batch API 50% discount on both input and output.

The enterprise cost stack (the real story)

Azure OpenAI on Global Standard matches OpenAI direct on model pricing. Enterprise deployments cost 15-40% more because of the supporting infrastructure:

Real workload cost comparison

1M requests/month × 100 input + 500 output tokens each on GPT-5.6 Terra:

The 15-40% premium isn't Microsoft marking up tokens — it's the infrastructure layer that turns a Chat Completions call into a production enterprise deployment. Whether that premium is worth it depends on your compliance and integration requirements.

EA / MCA / MACC integration

The biggest cost lever on Azure OpenAI at enterprise scale is bundling with your Azure commit. If your company has an Enterprise Agreement (EA) or Microsoft Customer Agreement (MCA), Azure OpenAI consumption rolls into it. Multi-year Azure spend commitments (MACC) include Azure OpenAI — so your OpenAI bills count against your committed Azure spend, and any negotiated MACC discount carries through.

In practice: enterprise customers with EA/MACC often get 10-30% off list Azure OpenAI rates depending on commit size. This can more than offset the infrastructure premium, making Azure OpenAI cheaper than OpenAI direct on a per-token basis for enterprise buyers.

Cost anti-patterns on Azure OpenAI specifically

• Using Regional Standard when Global Standard would meet compliance — pay premium for nothing
• Over-provisioning PTU during pilot, never measuring utilization once in production
• Provisioning PTU at 15 (Global minimum) when actual sustained load is 3 PTU worth
• Using On Your Data for tiny document sets that don't justify AI Search costs
• Logging 100% of prompts to Log Analytics at enterprise volume — massive ingest bill
• Creating separate resources per environment (dev/staging/prod each with a private endpoint) when one resource with multiple deployments would suffice
• Ignoring EA/MACC integration — many teams pay list rates their contract doesn't require

For cross-provider cost patterns spanning Azure OpenAI, OpenAI direct, Bedrock, and Gemini, see the AI API Cost Optimization Playbook 2026.

Azure OpenAI vs OpenAI direct

Every OpenAI model you can access on Azure exists on OpenAI direct. When does Azure win? When does going direct win? This chapter is the decision framework — split by factor because the answer varies.

Feature availability

Enterprise features

The commercial dimension — EA / MCA / MACC

If your company already has an Enterprise Agreement or Microsoft Customer Agreement, Azure OpenAI rolls into it. This is the single biggest reason enterprises pick Azure OpenAI over OpenAI direct:

• Consolidated billing — one bill, one procurement relationship
• Existing negotiated discount rates apply to Azure OpenAI consumption
• MACC commits count Azure OpenAI toward your spend commitment
• Resource-group-based cost attribution is cleaner than OpenAI's per-org/per-key model
• Partner programs (Microsoft for Startups, ISV, AI Co-Innovation) sometimes include Azure OpenAI credits

In practice, EA-covered customers often net a 10-30% effective discount vs OpenAI direct list rates, which more than covers the enterprise infrastructure premium described in Chapter 10.

Decision matrix — when to use which

The migration lift between platforms

Both platforms use the same OpenAI SDK — migration between Azure and direct is mostly a client configuration change plus deployment-name-vs-model-ID handling. Non-trivial parts of migration:

• Auth model — API keys vs Entra ID; refactor auth plumbing if switching to Entra ID
• Content filter behavior — Azure filters may over-block or under-block vs OpenAI's moderation; retest content moderation
• API version pinning — Azure requires it, direct doesn't; adjust request construction
• Feature parity — verify every feature you use is available on the destination platform
• Monitoring / logging — Azure integrates with Application Insights, direct doesn't

Many teams run both — Azure OpenAI for the bulk of enterprise workloads, OpenAI direct for feature-forward development and workloads not on Azure. LiteLLM or a similar routing layer makes this transparent.

Production checklist

Twenty items to verify before your Azure OpenAI integration goes to production. Each corresponds to at least one incident from real 2026 deployments — with attention to the Azure-specific issues (deployment naming, Entra ID, PTU utilization, content filters) that don't apply on OpenAI direct.

Auth & permissions

1. Entra ID auth used over API keys wherever possible; managed identity for service-to-service
2. Cognitive Services OpenAI User role assigned to every identity that needs it
3. API keys (if used) stored in Azure Key Vault, never in source code or environment variables committed to Git
4. Token scope verified — https://cognitiveservices.azure.com/.default for OpenAI, not other Azure services

Deployments & endpoints

5. Deployment names follow a naming convention that survives model swaps (e.g. purpose-based, not model-version-based)
6. Every SDK call passes the deployment name, not the model ID, in the model parameter
7. api-version pinned consistently across environments; version bumps tested before rolling out
8. Deployment type (Global/Data Zone/Regional) matches actual compliance requirement — no unnecessary Regional

Rate limits & PTU

9. TPM quota sized against expected peak, not average — build in headroom
10. Dynamic quota enabled on Standard deployments that need burst capacity
11. PTU utilization measured monthly; downsized if consistently below 70%
12. PTU reservations reviewed against cross-model catalog benefit (2026 change)
13. Circuit breaker for sustained 429s — fall back to alternate deployment or model

Content filters

14. Content filter thresholds tested against your actual user content — not just defaults
15. Both prompt_filter_results and content_filter_results checked in every response handler
16. Jailbreak detection logged to security monitoring, not treated as a generic filter block
17. Custom content filter policy created for legitimate use cases the defaults over-block

Networking & observability

18. Private endpoint configured for enterprise deployments; DNS resolves to private IP not public
19. Application Insights logging sampled (not 100%) at high volume to avoid ingest bill surprise
20. Request/response IDs logged for every call — required for Microsoft support cases

Frequently asked questions

The full library — 33 Azure OpenAI error pages

Every Azure OpenAI error-fix page on AI Error Hub, organized by category. Bookmark this section — when you hit a specific error, jump straight to the page that walks through it with tested code.