Budgets and Cost

Every metered call SOIT makes leaves a cost entry in the runtime ledger. Cost attribution reads those entries to say where spend went; budgets read them to limit it. Budgets and daily usage aggregates shipped in v1.2.0. v1.4.0 adds priced tool calls and image prices by quality and size, and changes how budgets hold calls in flight. v1.6.0 lines the ledger up beside a provider's bill (see Checking costs against a bill) and lets a workspace refuse calls with no price (see Calls with no price).

Where cost is recorded

• Each model call records its step, tokens, latency, and a cost entry (currency, amount, unit, quantity, provider, model or tool ref, token breakdown), whether it came from an agent, a workflow, or the gateway.
• Runs record the entry they came through (source: platform or gateway) and the API key that started them (api_key_id). A key issued to a service principal attributes runs and costs to the principal.
• A call through a virtual model is attributed to the provider that answered, or, for an image call that timed out, to the provider it was waiting on.
• Tool calls pass the same credit and budget checks as model calls. From v1.4.0 a tool can also declare a price; see Priced tool calls.
• Rehearsal runs, such as publish gates and model replays, are kept out of spend.
• New in v1.4.0. A cost entry without a price says why in pricing_snapshot.reason, such as pricing_not_configured for a model with no configured price, or no_price_source when nothing names a cause. pricing_snapshot.priced always matches whether the entry has an amount, and an amount of 0 with a currency is an explicit zero price.
• New in v1.6.0. Cost entries of model calls keep the provider's own ids, when it gave them: upstream_id, the response's id (chatcmpl-…, msg_…, Gemini's responseId), and upstream_request_id, the request id from the provider's response headers (x-request-id, request-id). The OpenAI, Anthropic, Gemini, and LiteLLM adapters read them for chat, streams, embeddings, reranking, and images. Neither id is unique, since self-hosted servers and caching proxies repeat ids, and an id an SDK made up is not kept (LiteLLM's own chatcmpl-<uuid>, and any id for Anthropic through LiteLLM, where only the request id survives). Embeddings and images have a request id at most; charges for unanswered image calls, tool calls, and entries written before v1.6.0 have neither.

Priced tool calls

New in v1.4.0. A tool can declare what one call costs in its ToolSpec policy:

"pricing": {"currency": "USD", "call": "0.002"}

Every successful call of the tool — through POST /api/v1/tools/{ref}/invoke, MCP tools/call, an agent, or a workflow — then writes a priced cost entry that counts against budgets and deducts credits like a model call. "call": "0" is an explicit free price. A call that is not charged keeps an unpriced entry whose pricing_snapshot.reason says why:

An agent's call to a tool of an enabled MCP server follows the tool's policy from the workspace catalog, as a direct call does: its declared price, and its approval rule. Earlier versions refuse a ToolSpec with pricing and their plugin loader skips such a tool, so once a plugin declares a price, do not roll back past v1.4.0.

Image prices

New in v1.4.0. A model's pricing_json prices images per image. image is the plain rate; image_variants lists prices for a quality, a size, or both, for providers whose price depends on them:

{
  "currency": "USD",
  "image": "0.042",
  "image_variants": [
    {"quality": "low", "price": "0.011"},
    {"quality": "high", "price": "0.167"},
    {"quality": "high", "size": "1536x1024", "price": "0.25"}
  ]
}

• A call is priced by the variant that fits it, that is whose every named value equals the call's. Of several, the one naming more values wins, and of equals the first listed.
• The cost entry's pricing_snapshot names the variant (image_variant) and records the call's size and quality under quantities.
• A call no variant fits takes image. Without an image rate it is recorded unpriced with the reason image_variant_not_priced, rather than billed at another quality's price.
• A call that asks for no quality, or for auto, is priced as one that names none.

How quality reaches the provider is described under Image options on the gateway page.

Charges SOIT estimates

New in v1.4.0. Most cost entries record what the provider reported. Two kinds of call are charged an estimate instead, because the provider was asked and its full answer never came, and SOIT giving up does not cancel the work at the provider:

• A streamed model call that ends without the provider's usage (the client left, the stream failed part way, or the backend never sends usage) is charged tokens estimated from the prompt and the text generated.
• An image call that times out after the provider was asked, or that was in flight when the process making it was lost, is charged the number of images it asked for, at its route's per-image price for the size and quality it asked for.

Both are flagged usage_estimated on the step's metrics and in the cost entry's pricing_snapshot, whose usage_estimate_basis says what the estimate counts (characters or requested_images). They count toward credits and budgets like any other call.

An image job lost with its process is failed with IMAGE_JOB_INTERRUPTED by the image job reaper, which is off unless IMAGE_JOB_REAPER_ENABLED is set; the shipped compose files and .env.example set it. An image call in progress marks its run alive every IMAGE_JOB_HEARTBEAT_SECONDS (30), and a run left unmarked for IMAGE_JOB_ORPHAN_AFTER_SECONDS (600) counts as lost. A job lost before it asked the provider is not charged. During a rolling upgrade, enable the reaper only once every replica runs v1.4.0: earlier versions send no heartbeat.

Reading spend

• Observe › Runs has an Entry filter and a key filter, and the cost overview follows them. Settings › API links each key to its runs.
• The run list (GET /api/v1/runs), its CSV export, and every cost breakdown under /api/v1/runs/costs filter by source and api_key_id, so gateway traffic and its spend can be read on their own or per key.
• New in v1.6.0. Observe › Costs shows the ledger's spend for a window, ready to set beside a provider's bill; see Checking costs against a bill.
• New in v1.6.0. Costs in different currencies are no longer added together. The Overview spend tile shows one figure per currency, names the currency in its label when there is one, and gives a change against the previous window only when both windows are in the same currency. A run's dashboard cost is left out when the run is priced in more than one currency, and the run detail page totals each currency apart. Unpriced entries add nothing, so a run with only those shows a dash, not zero.

Checking costs against a bill

New in v1.6.0. Every metered call leaves one cost entry, so the ledger can be read the way a provider's bill is: per key, per principal, per model or tool, per day. Each entry has a pricing status:

An unpriced entry is never counted as zero, and amounts in different currencies are never added together.

GET /api/v1/runs/costs/entries?since=…&until=…&api_key_id=…&pricing_status=unpriced
GET /api/v1/runs/costs/reconciliation?since=…&until=…&group_by=model

• Both take the same filters: the run's api_key_id, user_id (a member or a service principal), and source (platform or gateway), and the entry's model_ref, provider_slug, tool_ref, source_port, operation, currency, and pricing_status. upstream_id and upstream_request_id filter too, so a line of a provider's request log finds its entry.
• /costs/entries lists entries oldest first, each with its run's api_key_id, user_id, and run_source, its pricing_status, and, when unpriced, its unpriced_reason. Its until is inclusive, as before. It pages with page_token and page_size (200 by default, at most 500).
• /costs/reconciliation covers the half-open window [since, until), like the ledger exports. It answers the priced total per currency (amounts), the part of it computed from estimated usage (estimated_amounts), the entries per status (status_counts), and the unpriced entries per reason (unpriced_reasons). With group_by (model, provider, tool, api_key, user, source, operation, or day) it adds one row per value and currency, with unpriced entries in a row of their own; at most 500 rows, and groups_truncated says when there were more.
• external_reconciliation is always not_performed: this is the ledger's side of the check only.
• Entries are dated when they were written. A charge the image job reaper records for a lost call is dated when the reaper ran, so it can fall in a later window than the call.
• Providers' usage exports are mostly per hour or day, so most bills are still matched by model, key, and time window; the provider ids help where a provider's request log is at hand.
• Workspace readers can call both, as they can the other /api/v1/runs/costs reads.

In the console, Observe › Costs shows the reconciliation for the last 24 hours, 7 or 30 days, or this month: one figure per currency with the estimated part apart, the unpriced entries counted and listed by reason, and a table grouped by model, provider, tool, API key, principal, or day, filtered by pricing status and source. Export CSV downloads the grouped rows to set beside a bill. The page says the provider's bill itself has not been checked.

Daily usage aggregates

The usage_daily_aggregates table sums metered calls, tokens, and priced amount per UTC day, source, user or service principal, key, provider, model, and operation. Rehearsal runs are kept apart as source rehearsal.

• A cost.recorded consumer adds each usage fact as it arrives, exactly once.
• A reconciler in the outbox dispatcher process rebuilds each finished day from the cost ledger, every USAGE_RECONCILE_INTERVAL_SECONDS (hourly by default).
• The two never count a fact twice.

Budgets

A budget limits spend for one scope, per UTC day or month, in one currency:

Workspace owners and admins manage budgets under Govern › Budgets or with /api/v1/billing/budgets. The console lists every budget with its spend in the current period against the limit, its forecast, when it resets, and what happens at the limit, and flags the ones at risk, spent, or on course to overrun.

Spend is read from the daily aggregates for completed days and from the cost ledger for today. GET /api/v1/billing/budgets/{budget_id}/status reports one budget's spend, remainder, percentage, and forecast to the end of the period; GET /api/v1/billing/budgets/statuses reports every budget in one call.

Hard stop

A budget with a hard stop refuses model and tool calls once spent, with 402 naming the budget, what it spent, and when it resets. The refusal is written to the audit ledger.

The gateway also answers 402 (insufficient_quota) when credit is exhausted.

Calls with no price

New in v1.6.0. A call to a model or tool with no price configured is recorded without an amount, so no budget counts it and a hard-stop budget cannot stop it. The workspace setting Calls with no price in Settings › Security (unpriced_call_policy on PATCH /api/v1/workspaces/{workspace_id}, changed by workspace admins) decides what such a call does:

• A refused call is answered before the provider or the tool is called, naming the model or tool: 403 with code PRICING_NOT_CONFIGURED on /api/v1, and 403 permission_error with code pricing_not_configured on the /v1 gateway.
• The refusal is written to the audit ledger as billing.unpriced.blocked.
• A virtual model moves on to a priced target.
• A refused model call gives back the budget hold its budget check took, instead of keeping it until the hold expires.
• An image is priced by its size and quality, so a model priced for one request can be unpriced for another.
• To keep a local model callable under either refusal, give it an explicit zero price ({"currency": "USD", "input": 0, "output": 0, ...}).

Holds for calls in flight

Each admitted call holds its budgets for the period's average call cost until its cost is committed, so concurrent callers overshoot a limit by about one call at most. Holds live in Redis.

New in v1.4.0:

• Checking a budget and holding it are one Redis step, so two concurrent callers can no longer both take the last slot.
• A hold is released as soon as the call's cost is committed, instead of lasting a minute, so a busy workspace is no longer refused near its limit for calls that already finished.
• A failed call's hold is released when its run ends. A hold that nothing releases lasts the call's timeout and a minute, so a long image call keeps its budget held to the end.
• While Redis is unreachable, each process keeps its own holds instead of skipping them.

In v1.3.0 a hold expired after about a minute rather than being released when the call finished, and holds were skipped without Redis.

Threshold alerts

Crossing a threshold (50, 80, and 100 percent by default) publishes one billing.budget.threshold_reached event per budget, period, and threshold. Workspace owners and admins are notified in their inboxes, on their own email and webhook endpoints as their delivery preferences allow, and on the workspace's team channels subscribed to the alert category. Team channels are managed under Settings › Notifications or with /api/v1/notifications/workspace-endpoints.

New in v1.4.0. Each cost is judged against the spend as of that cost, so a threshold is announced even when the outbox dispatcher runs behind. Before, a threshold was judged against the spend when the cost event was handled, so costs recorded meanwhile could carry spend past a lower threshold before its event was read, and that threshold was never announced.

Aggregates, thresholds, and alerts are worked out by a process with the default role. A gateway-only process runs none of that background work, so at least one process with the default role must run beside it.

Evidence of spend

• Cost entries leave SOIT in the versioned ledger contract through GET /api/v1/exports/costs, with amounts as decimal strings so no reader rounds them.
• New in v1.6.0. The contract is version 1.1: cost records may carry the optional upstream_id and upstream_request_id. Evidence bundles and ledger exports declare 1.1, so a consumer that rejects unknown fields must accept the two.
• A run's evidence bundle includes its cost records (costs.jsonl).
• Budget refusals are audit events, so the audit export includes them.

API and UI surfaces

Console: Govern › Budgets, Observe › Runs, Observe › Costs, Settings › API, Settings › Security, Settings › Notifications.

Known limitations

As of v1.6.0:

• Cost reconciliation lines the ledger up beside a bill; SOIT does not import or read provider bills, and the console says the bill itself has not been checked.

• A call that fails, or records no cost, keeps its budget hold until its run ends (for a gateway call, when the call returns), or at most its timeout (LLM_TIMEOUT_SECONDS, or LLM_IMAGE_TIMEOUT_SECONDS for images) and a minute. A run that never ends on the process that admitted the call releases the hold only when it expires, and a call still running after that, such as a very long stream, no longer holds its budget.

• While Redis is unreachable, replicas do not see each other's holds and can each overshoot a budget by about one call.

• Budget spend reads the daily aggregates for finished days and the cost ledger for today; a day's aggregate is rebuilt by the reconciler (USAGE_RECONCILE_INTERVAL_SECONDS, hourly by default).

Related pages

• OpenAI-Compatible Gateway — per-key limits and service principals.
• Evidence and Exports — taking cost records out.
• Notifications — where threshold alerts land.
• Tasks and Runs — the ledger cost entries belong to.