AI API Cost Attribution — Know What Every Feature Actually Costs

Why the monthly bill is not enough#

A provider invoice tells you the total. It does not tell you which feature drove the spend, which customer received value, or which route burned tokens on retries. Cost attribution fills that gap.

It starts with a stable request identity. Every model call should carry a request ID that links the application event, the gateway route, the provider response, the usage record and the billing entry. For workflows with several model calls, add a parent operation ID so the team can calculate the full cost of one user action.

With those IDs in place, every later question, from which feature is expensive to which tenant is unprofitable, becomes a query instead of a guess.

Choose dimensions and preserve token detail#

Define the dimensions that matter before collecting data. Useful labels include product, feature, environment, tenant, model, provider, region, endpoint type and workload class. Avoid email addresses, raw prompts or arbitrary user input in metric labels; use internal identifiers and keep sensitive mappings in controlled systems.

Attribute both input and output usage. Input can include system instructions, retrieved documents, tool schemas, conversation history and user content. Output can include reasoning, visible text, tool arguments or structured data. If a provider reports cached tokens or separate reasoning usage, keep those fields rather than collapsing everything into one total.

Decide these labels early. Adding a dimension later means historical data cannot be split the same way, and the reports that matter most are usually the ones comparing before and after a change.

Count the whole workflow, including retries#

A single support answer might involve embedding, retrieval, reranking, a chat completion and a follow-up tool call. Looking only at the final chat request understates the real cost. Aggregate every child operation under the user-visible outcome and record whether that outcome succeeded.

Retries and fallbacks need special treatment. A timeout can still generate provider usage even though the application received no answer, and the retry then creates a second charge. Record each attempt separately but attribute all of them to the same parent operation. That exposes the cost of unreliability instead of hiding it inside an average.

Use real route pricing and measure outcomes#

Model aliases can point to different providers or pricing tiers over time. Store the selected route, price version, currency and timestamp with each usage record. Recalculating historical cost with today's price table makes reports disagree with invoices and hides the effect of routing changes.

Then measure cost per successful outcome. Cost per request can reward cheap failures. Better metrics are cost per resolved support ticket, accepted code change, completed document, qualified lead or successful agent task. Pair spend with latency and quality so optimization does not quietly damage the product.

Budgets, leakage and shared costs#

Set budgets at several levels, such as organization, team, feature, tenant and experiment, and use soft alerts before hard limits. A cost spike can come from growth, a bug, a prompt expansion or an attack, and the right response depends on the cause, not only the number.

Watch for common leakage patterns, which often save more than negotiating a small price cut:

• Duplicated requests and retry storms
• Unbounded conversation history and repeated retrieval context
• Oversized tool schemas
• Accidental use of premium models
• Abandoned streaming responses
• Test traffic hitting production

Shared infrastructure such as gateway fees, storage, vector databases, observability and background evaluation rarely maps cleanly to one request. Separate direct model cost from allocated platform cost, and document the allocation rule so finance and engineering understand why a feature's margin changed.

From dashboards to routing decisions#

Give product teams a dashboard with daily spend, cost per successful outcome, model mix, retry cost, cache savings and the largest changes by feature. It should let a team move from an anomaly to representative operation IDs without exposing customer content.

Reconcile estimates with provider records. Usage events can be delayed, duplicated or missing after an incident, so compare internal totals with provider exports, gateway records, credits and invoice adjustments, and keep the difference visible rather than forcing the numbers to match.

Once cost and quality are tied to each workload, routing becomes an evidence-based decision: smaller models for routine tasks, premium models for difficult cases, and fallbacks only where the expected value justifies them. With an OpenAI-compatible gateway such as Crazyrouter, shifting a workload to a different model is a change to the model string, which makes acting on that evidence cheap.