AI API Error Taxonomy — Make Failures Actionable

Separate transport failures from operation failures#

When an AI API fails, the caller needs to know whether to retry, change the request, wait for work that may still be in progress, or accept that the gateway is protecting a budget or policy. A generic status code answers none of that.

Start by splitting two dimensions. Connection resets, DNS errors, TLS problems and gateway timeouts describe communication. Invalid requests, unsupported model capabilities, policy denials and exhausted quotas describe the operation. Keep them separate even when they share an HTTP status: retrying a malformed request only creates noise, while retrying a transient connection error may recover.

Status codes are a transport hint, not the full taxonomy. A 400 can be a fixable schema error or an unsupported capability. A 401 can be an expired gateway credential, while a provider credential failure is an operator incident. A 429 can be a tenant quota, a provider rate limit or global overload. Put the precise category in a stable code.

A stable public error envelope#

Each error response should carry:

• an error type and a stable, documented code such as invalid_request, rate_limited, provider_unavailable or policy_denied
• a human-readable message
• a request ID
• a retryable hint, when that hint is safe to give

Provider codes or upstream statuses belong in structured metadata, not as the whole contract. Clients should branch on documented codes rather than parsing prose that can change with any deployment.

Make ownership explicit too. The client owns malformed JSON, missing fields, invalid parameters and expired credentials. The gateway owns routing policy, tenant limits, request validation and shared capacity. The provider owns its availability and model-specific limits. Clear ownership speeds up support and stops teams from retrying problems that only a code or config change can fix.

Classify errors by the next safe action#

Fixable client errors need a corrected request. Authentication and authorization errors need a credential or permission change, not a retry loop. Rate limits need bounded waiting that respects Retry-After. Capacity or provider availability errors may justify a controlled fallback. Policy errors need a different allowed operation or a review path. Unknown outcomes need reconciliation before chargeable work is repeated.

Retryable does not mean safe to retry. A transient failure can still be dangerous to repeat if the provider may have accepted the request or a tool may have produced a side effect. Base retry decisions on operation state, idempotency information and provider certainty, and do not promise exactly-once behavior you cannot prove.

Internally, preserve the whole chain: gateway classification, adapter result, provider status, timeout stage, attempt number and final action. A provider 503 may surface publicly as provider_unavailable, but the trace should show whether it arrived before headers, during streaming, after partial output, or after a queued job was accepted.

Streaming, async jobs and fallbacks#

Streaming errors depend on stage. Before the first token, return a normal structured error. Once the stream has begun the HTTP status cannot change, so send a typed terminal event with the request ID, whether output is partial, and whether the operation may still be running. Clients must tell clean completion, cancellation, provider failure and interrupted connections apart, and partial output must never be labeled complete.

Async jobs need durable failure state. Webhooks should identify the job, attempt, error code and whether the job can resume or must be recreated. Store the last failure and a safe next action so polling clients do not infer state from a missing event. If a provider accepted a job but the callback never arrived, reconcile using the provider operation ID before declaring failure or starting a duplicate.

Fallbacks should follow the classification. They make sense for provider_unavailable or a bounded timeout when another model is compatible, and are usually wrong for invalid_request, policy_denied, authentication failures or gateway-caused schema mismatches. Check capability, data residency, budget and quality first, and log both the original and fallback classifications.

Safe messages, contract tests and versioning#

Never echo API keys, authorization headers, full prompts, private tenant names or sensitive provider payloads. Redact model inputs in ordinary logs, cap upstream message size, and give callers a request ID plus a support-safe explanation while operators use a protected trace view.

Treat the taxonomy as a tested contract. Build fixtures for malformed requests, unsupported parameters, expired credentials, tenant limits, provider 429s and 5xx responses, DNS failures, timeouts before and after acceptance, invalid model output, policy blocks, stream interruptions and duplicate webhooks. Assert code, status, retry hint, headers, trace fields and billing behavior, or the document will drift as adapters change.

Measure error rate by code, endpoint, route, provider, tenant tier and client version, and alert on shifts such as invalid_request turning into provider_unavailable, new unknown codes, or more errors after provider work was consumed. Keep codes stable, add fields compatibly, and map old provider-specific errors rather than exposing raw provider codes. Translating those differences into predictable behavior is a core job of a gateway like Crazyrouter.