Skip to main content

Budgets and pricing

A budget caps model spend in USD for a directory user or group. Before AI Gateway v2 calls a provider, it asks the budgets service in Enterprise Manager to admit the request. After the response, it reports the provider's usage so the service can charge it.

Use Budgets in the console for routine administration. Use the REST API for automation, bulk import, and pricing. Budgets belong to users and groups, so the same budgets apply to traffic from both the AI Gateway and AI Gateway v2.

Cover every caller before you connect the gateway to budgets

A connected gateway refuses every caller without an applicable budget. Use one of the approaches in the next section before you set budgets.endpoint on a gateway that serves traffic.

Cover every caller first​

Before enabling budget enforcement for an environment with traffic, choose one of these approaches:

Set an organization default. This covers directory users without an explicit budget. Set it on the Defaults tab or through the API.

Create budgets individually. Use the bulk import for an existing user and group inventory.

Create budgets​

Each budget has a USD limit and a daily, monthly, or yearly calendar period. The budgets endpoints are part of the Enterprise Manager API.

The API identifies a budget by scope and subject. scope is user or group. subject_id is the user's directory UUID or the directory group's UUID. Find a user's UUID in the id field that GET /v1/users returns. That endpoint accepts an email filter. limit_usd is a decimal string and period is day, month, or year.

Set an organization default​

Set the organization default on the Defaults tab or through the API:

curl -X PUT https://<ENTERPRISE_MANAGER_URL>/v1/budgets/default \
  -H "Authorization: Bearer <ADMIN_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"limit_usd": "350.000000000", "period": "day"}'

Setting the default replaces its previous value. Deleting it leaves users without explicit budgets uncovered. Moving a user between the default and an explicit budget preserves their current-period usage.

Update one budget​

Use the user or group endpoint to create or update one budget:

curl -X PUT https://<ENTERPRISE_MANAGER_URL>/v1/budgets/users/<USER_UUID> \
  -H "Authorization: Bearer <ADMIN_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"limit_usd": "5.000000000", "period": "day"}'

curl -X PUT https://<ENTERPRISE_MANAGER_URL>/v1/budgets/groups/<GROUP_UUID> \
  -H "Authorization: Bearer <ADMIN_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"limit_usd": "500.000000000", "period": "month", "precedence": 1}'

Group budgets use 1-based precedence. The gateway charges an explicit user budget first.

Import in bulk​

PUT /v1/budgets replaces the entire collection: budgets in the request are created or updated, and budgets absent from the request are deleted. Use it only for complete imports.

curl -X PUT https://<ENTERPRISE_MANAGER_URL>/v1/budgets \
  -H "Authorization: Bearer <ADMIN_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "budgets": [
      {"scope": "user",  "subject_id": "<USER_UUID>", "limit_usd": "5.000000000",   "period": "day"},
      {"scope": "group", "subject_id": "<GROUP_UUID>", "limit_usd": "500.000000000", "period": "month"}
    ]
  }'

Bulk replacement preserves spend counters. A bulk import takes no precedence field. Group budgets are charged in the order their entries appear in the request, and the import renumbers every group, replacing any precedence you set through the per-group endpoint.

GET /v1/budgets returns the collection with an ETag. Send it back as If-Match on your next write to detect a concurrent change; a 412 response means re-read and retry.

Which budget gets charged​

The gateway charges budgets in this order:

  1. The user's explicit budget, or the organization default.
  2. The first group budget with remaining capacity, ordered by ascending precedence.

The gateway denies the request when no applicable budget has capacity.

Connect the gateway to budgets​

AI Gateway v2 reaches the budgets service through hook entries that the chart renders from the budgets values:

Hook entryWhen it runsWhat it does
budgets-admissionBefore the provider callAdmits or denies the request. A failed call denies the request.
budgets-usageAfter the provider responseRecords usage and charges a budget. The client receives the response only after it does.
budgets-captureAfter the other request-time hooksForwards the conversation for capture. Best effort, and never blocks the request.

The chart renders budgets-capture only when budgets.capture is true. Conversation capture is experimental. Enterprise Manager accepts captured conversations only when global.stacklok.experiments.insightsClassification.enabled is true, subject to a release-channel guard. Without it, budgets-capture calls fail and its probe reports errors, while admission and usage are unaffected.

Set these values to connect the gateway:

ValuePurpose
budgets.endpointHTTPS base URL of the Enterprise Manager budgets receiver, for example https://<ENTERPRISE_MANAGER_SERVICE>.<NAMESPACE>.svc:443.
budgets.tokenProjection.enabledProjects a ServiceAccount token that the receiver validates. Required when endpoint is set.
budgets.tokenProjection.audienceAudience of that token. Defaults to budgets-webhook and must match enterprise-manager.webhookAuth.audience.
budgets.caBundle.secretNameSecret holding the CA bundle that verifies the receiver's certificate. Falls back to global.webhooks.caBundleSecret.
budgets.sessionHeadersSession-correlation headers the gateway forwards. Inherits global.stacklok.sessionHeaders when empty.
budgets.captureForwards request bodies to the budgets service for conversation capture. Defaults to false.
budgets.acknowledgeUnmeteredSet to true to install a gateway with no endpoint. That gateway admits every request and charges nothing.

The chart refuses to render when endpoint is empty without acknowledgeUnmetered, when endpoint isn't an https:// URL, when token projection is off, or when no CA bundle is configured. For the full set of platform values and the order to apply them in, see Configure AI Gateway v2.

With budgets.capture on, the budgets service receives the request body the provider sees. Before you turn on capture, set up PII scanning to redact or block every entity you need to keep out of storage, and keep scanning.ner.failureAction at fail-closed. Capture stores a log-only match, and a request forwarded unscanned under fail-open, exactly as the caller sent it.

Verify the gateway reaches the budgets service​

At startup, the gateway pings each hook entry's receiver until it answers. When pings keep failing, the gateway logs a warning that names the hook entry and the failure reason, and logs receiver hook probe recovered once the receiver answers:

kubectl logs -n <NAMESPACE> deploy/<RELEASE_NAME>-ai-gateway-v2 \
  | grep 'receiver hook probe'

The same pings feed the stacklok.ai_gateway.webhook.probes counter, labeled by webhook_name, outcome (success or error), and webhook_reason. A failing ping usually means the token audience doesn't match the receiver's, the CA bundle doesn't trust the receiver's certificate, or budgets.endpoint points at the wrong Service. Pings don't affect readiness, so a pod reports ready while its budgets receiver is unreachable.

Update the pricing catalog​

Enterprise Manager ships a baseline catalog generated weekly from public model pricing, and each upgrade refreshes every rate you haven't changed. To override a price, update one model's entry at a time through the API. Changes take effect without a restart.

GET /v1/budgets/pricing returns the active catalog with an ETag header. Send that value as If-Match with a PUT to the entry's address, /v1/budgets/pricing/entries/<PROVIDER>/<MODEL>. Use the provider and model exactly as they appear in the catalog. A model name can contain /. The body carries exactly one rate block: anthropic, openai, or google.

curl -i https://<ENTERPRISE_MANAGER_URL>/v1/budgets/pricing \
  -H "Authorization: Bearer <ADMIN_TOKEN>"

curl -X PUT https://<ENTERPRISE_MANAGER_URL>/v1/budgets/pricing/entries/anthropic/claude-sonnet-4-6 \
  -H "Authorization: Bearer <ADMIN_TOKEN>" \
  -H 'If-Match: <ETAG>' \
  -H "Content-Type: application/json" \
  -d '{"anthropic": {"inputUsdPerMillion": "3", "outputUsdPerMillion": "15"}}'

The ETag identifies the whole catalog version, so a write to any entry changes it. A request without If-Match returns 428, and a stale value returns 412. Read the catalog again and retry.

Each write publishes a new complete catalog version, effective immediately. The entry you write stops tracking the baseline, so later upgrades leave its rate alone. Every other entry keeps tracking the baseline.

To withdraw or restore a rate, use these requests, which also require If-Match:

  • DELETE on the entry removes its rate. The gateway denies requests for that model until you price it again, and upgrades don't restore it.
  • POST to /v1/budgets/pricing/entries/<PROVIDER>/<MODEL>/revert returns the entry to the baseline's current rate. It returns 409 when the baseline doesn't price that model.

GET /v1/budgets/pricing/versions lists published catalog versions, and GET /v1/budgets/pricing/versions/<EFFECTIVE_FROM> returns one of them. To restore an earlier rate, read it from that version and PUT it on the entry.

Rate fields by provider​

Set provider-specific rates for cache creation, reasoning, audio, prompt-size tiers, and server-side tools when the model reports those usage categories. The examples below show entries as the catalog stores them. In a PUT body, send only the provider block. The provider and model come from the URL.

Anthropic reports cache-creation tokens split by time to live, so declare each rate separately:

{
  "provider": "anthropic",
  "model": "claude-sonnet-4-6",
  "anthropic": {
    "inputUsdPerMillion": "3",
    "cacheCreation5mInputUsdPerMillion": "3.75",
    "cacheCreation1hInputUsdPerMillion": "6",
    "cachedInputUsdPerMillion": "0.3",
    "outputUsdPerMillion": "15"
  }
}

OpenAI reasoning models report reasoning tokens, and the audio families report audio input and output separately:

{
  "provider": "openai",
  "model": "gpt-4o-audio-preview",
  "openai": {
    "inputUsdPerMillion": "2.5",
    "cachedInputUsdPerMillion": "1.25",
    "outputUsdPerMillion": "10",
    "audioInputUsdPerMillion": "40",
    "audioOutputUsdPerMillion": "80"
  }
}

Some models charge a higher rate above a prompt-size threshold. Encode that with promptSizeTiers:

{
  "provider": "google",
  "model": "gemini-2.5-pro",
  "google": {
    "inputUsdPerMillion": "1.25",
    "cachedInputUsdPerMillion": "0.125",
    "outputUsdPerMillion": "10",
    "thoughtsOutputUsdPerMillion": "10",
    "promptSizeTiers": [
      {
        "aboveTokens": 200000,
        "inputUsdPerMillion": "2.5",
        "cachedInputUsdPerMillion": "0.25",
        "outputUsdPerMillion": "15"
      }
    ]
  }
}

Server-side tools that bill per thousand calls appear in the catalog's tools block, per provider rather than per model. These fees come from the baseline catalog, and the API can't change them:

{
  "entries": ["...per-model entries..."],
  "tools": {
    "anthropic": { "webSearchUsdPerThousand": "10" },
    "openai": {
      "webSearchUsdPerThousand": "10",
      "fileSearchUsdPerThousand": "2.5"
    },
    "google": { "groundingSearchUsdPerThousand": "35" }
  }
}

Match provider kinds to catalog entries​

Each catalog entry is keyed on a provider and a model. The catalog provider isn't always the same string as the AI Gateway v2 provider kind:

Provider kindCatalog provider
anthropicanthropic
bedrock-anthropicawsanthropic
openaiopenai
openrouteropenrouter
gemini-aistudiogoogle
vertexgoogle

The model in a catalog entry must match the model string in providers[].models exactly. A dated model ID and its undated alias are two different catalog entries. The gateway doesn't check its route table against the catalog, so confirm that a model is priced before you add it to a provider.

Monitor spend​

Use Budgets to compare usage with each limit.

Use the API for finance reports and external dashboards:

EndpointReturns
GET /v1/budgets/summaryUsed, limit, and remaining per principal for the period
GET /v1/budgets/spendA spend time series, for charting
GET /v1/budgets/breakdownCurrent-period spend for one budget, by user, model, or provider
GET /v1/budgets/chargesRecorded charges, newest first, with the exact amount charged
GET /v1/budgets/eventsRaw usage events, including those that resolved to no charge

GET /v1/budgets/groups/<ID> and GET /v1/budgets/users/<ID> return the same breakdown addressed per budget, with a dimension query parameter.

Any authenticated caller can read their own events, charges, summary, and spend series at /v1/me/budgets/events, /charges, /summary, and /spend without an admin grant. The admin routes above require the platform admin grant, the same one directory administration requires; there is no separate budgets role.

How refusals reach the client​

The budgets service decides admission, and the gateway returns its status to the client with a request denied by governance hook message:

StatusCause
402Every applicable budget is exhausted (budget exhausted), or no budget covers the caller (no applicable budget).
403The caller doesn't resolve to a directory user, the directory couldn't be reached, or the model has no entry in the active pricing catalog. The message names the budgets-admission hook.

For an unpriced model, Enterprise Manager logs budget admit: model not priceable by the active catalog; denying with the model and catalog provider, so you can add the missing entry.

When the gateway gets no usable answer from the budgets service, it denies the request:

StatusCause
503The budgets service is unreachable.
504The call timed out. Each attempt must answer within 650 milliseconds, and the gateway retries a failed attempt once within a 5-second limit.
502The budgets service returned an error status or a malformed reply.

The gateway reports usage once per interaction, after the provider response completes. For a non-streaming request, the gateway holds the response until budgets-usage records it. If the usage report fails, the client receives a 502 error, by default metering unavailable; interaction was not recorded.

For a streaming request, the gateway sends every frame except the last one as it arrives and holds the last frame until budgets-usage records the interaction. If the usage report fails, the gateway replaces the last frame with an error event in the client's API format, so the client's parser reports an error rather than a clean finish. On Gemini streams the last frame carries the final tokens, so a failed report also withholds them.

When a budget runs out​

An exhausted budget returns 402 before the gateway calls the provider. The error message tells an exhausted budget (budget exhausted) apart from missing coverage (no applicable budget). Check the caller in the console to see which budgets apply.

Access resumes when the calendar period resets. To restore access immediately, raise the limit. The change applies on the next request and preserves the usage counter.

Next steps​