Development of pay-per-inference billing for AI models

Pay-per-inference billing: where model call accounting breaks

Counting model calls is easier than billing for them. A counter records the fact of a request, while cost is determined by input context and response length: the same call with a long input and short response can differ in consumption by several times.

When price is based on the number of requests, inference billing stops reflecting load — heavy scenarios are charged at the price of light ones.

Model call accounting breaks on retries and streaming delivery. The response was cut off midway — the request has already been counted, the client repeats the request, and the log contains two records where there was one operation.

Batch processing and cached responses add their own issues: one request produces several results but is billed as a single unit.

Limits and quotas require reservation, not post-factum counting.

While consumption is charged after the model response, parallel requests from one client pass the check simultaneously and push the client over the threshold; the prepaid balance goes negative, and the difference has to be charged retroactively or written off.

Then disputed charges begin. Without a log where each request has an identifier, model version, input and output volume, and retry flag, invoice discrepancies cannot be substantiated — both sides rely on words. We explain how this is closed.

Measurable effect: charge accuracy and downtime

The effect of billing rework is measured not by feelings but by a set of checkpoints — we capture them on staging before launch and in production afterward. Below is the list by which we accept the work: each item is either done or not, and this is visible in reports.

• Charge accuracy — the billing log is reconciled with actual model calls: discrepancy within fractions of a percent, unaccounted calls remain isolated cases rather than systemic loss.
• Share of disputed charges — operations that cannot be matched to an invoice are kept below 1% and resolved by an automated rule, without manual recalculation.
• Period close — instead of multi-day exports, the report closes in hours: aggregates are calculated as events arrive.
• Peak load — a multiple-fold increase in the call flow does not lose events: ingestion and calculation are separated, so the spike is stretched rather than turning into a hole in the invoice.
• Retries — idempotency by request key: a client retry or connection drop does not lead to double charging, duplicates are filtered out at ingestion.
• Billing downtime — when a node fails, charges are accepted into a buffer and arrive after recovery; the data loss window is measured in minutes, not hours.
• Observability — deviations and event processing speed are visible in Grafana, and an alert arrives before the client notices a discrepancy in the invoice.

Pay-per-inference billing models: comparison of approaches

Any billing model relies on a single accounting layer: each model call writes an event — tenant, model, volume of consumed tokens. A tariff is the rule by which events turn into charges.

Therefore, the mode is chosen not by billing convenience but by the shape of the load and by how many reconciliation discrepancies the customer is willing to tolerate.

Modes rarely exist alone: a subscription covers the base volume, while calls over the limit go to a separate tariff. Below is a comparison by load and reconciliation: where a mode behaves predictably and where it starts to get in the way.

All modes share one common limitation: a charge is no more accurate than the event log beneath it. Duplicate delivery, lost response, client-side retries — all of this is either suppressed by an idempotent call key or turns into discrepancies.

Stages of implementing the billing pipeline

Pay-per-inference billing breaks not on the model but on accounting: retries, canceled streams, and partially delivered responses turn into disputed lines in the invoice. Therefore, we start with an inventory of events, not with code.

1. Accounting audit. We collect event sources — gateway logs, model metrics, access logs — and record where the fact of a request is lost and how the client is identified. The output is an accounting map with a list of gaps.
2. Billing scheme. We choose the unit: tokens, calls, or inference time; define rounding, tiers by model, limits, and behavior when they are exceeded. We agree on the scheme in a table before writing code.
3. Pipeline setup. We install counters with idempotency by request identifier, aggregate into billing periods, and bind the tariff to the client. We process retries and cancellations so that one call does not appear in the invoice twice.
4. Test runs. We run synthetic and shadow traffic, reconciling billing amounts with raw logs. Edge cases: stream interruption, mid-stream cancellation, resend, empty response.
5. Reconciliation act. We reconcile the calculation with actual consumption for the period and resolve discrepancies by source down to zero. After the act is signed, the numbers can be trusted.
6. Handover of regulations. We hand over the billing scheme description, discrepancy investigation procedure, anomaly alerts, and access. The customer's team resolves typical issues on its own.

Accounting schemes, limit policies, and documentation

Work on pay-per-inference billing ends not with launch but with handover of the pipeline: the customer must be able to change limits, read charges, and explain discrepancies on its own.

Therefore, everything we design remains in artifacts — from the accounting scheme to the reconciliation regulation.

• Accounting scheme: what counts as a call, where input and output are recorded, how a streaming response is correctly closed on timeout or interruption.
• Tariff configuration: binding a model to a unit price, tariff versioning, model directories, rounding rules.
• Limit policies: thresholds per key, team, and project; behavior on exhaustion — soft rejection or degradation; exceptions for internal tasks.
• Reconciliation regulation: how provider counters and ours are matched, where the discrepancy log is stored, who closes an incident and within what timeframe.
• Monitoring dashboards and alerts: spend by project, failure rate, deviation of actual consumption from expected.
• Support instructions: how to find charges by key, reissue a key, what to do in case of anomalous spend growth.
• Role and access matrix: who changes limits, who sees finances, who works only with logs.
• Pipeline handover: repository, data schemas, migrations, environments, and the procedure for rolling out changes.

Separately, we hand over a decision log — why this accounting granularity was chosen, what trade-offs were made on idempotency and streaming responses. The new team does not reopen these debates and makes billing changes according to documented rules.

Case study: how discrepancies in charges were eliminated

What was

The platform provided access to several models: text generation, recognition, and vector embeddings. Accounting was based on events from execution services, and when a request was retried after a timeout, one request appeared in the invoice twice.

Charge discrepancies reached 4% of requests, the share of disputed invoices was about 5%, and support performed reconciliation manually: two to three days at the end of each billing period.

What was changed

The event began to be formed at a single point — at the gateway before the model call — and went into a stream on Kafka.

Each call received an idempotency key, and the accounting service writes to the log in append-only mode: a retry with the same key is discarded rather than summed.

The tariff and token volume are recorded in the same event, so the invoice amount is reconstructed from primary data.

What became

Charge discrepancies fell below 0.1% of requests, and the share of disputed invoices dropped to 0.4%.

Automated reconciliation runs daily, takes 10-15 minutes, and raises an alert when the deviation exceeds the threshold: an error is visible the next day rather than at period close.

The failure class "double charge on retry" is closed by the data schema — the handler cannot apply the same operation twice.

Before changing the scheme, billing ran in shadow mode for a month: the old and new calculations ran in parallel, and discrepancies were investigated line by line.

Most invoice-related support requests disappeared — a typical question is resolved from the call log in a couple of minutes.

What to do if billing already runs on custom accounting?

Custom accounting is almost never rewritten from scratch — it is extended into a managed pipeline.

Usually it already collects an event stream and aggregates, but it lacks resilience to retries, clear reporting period boundaries, and the ability to recalculate a closed month — hence the accumulated discrepancies.

The first step is to freeze the current logic as a reference and capture its behavior on historical data.

The key is to keep the usage log separate from aggregates: raw events with key, model, input and output consumption, cache-hit flag, and retry flag allow any amount to be recalculated and explained.

Then discrepancy reconciliation happens not "at month end" but for each line item, and each one has a cause: double accounting on retry, period boundary shift by time zone, lost consumption on a streaming response, rounding during aggregation. For pay-per-inference billing, this is the source of truth.

Data migration is divided into reference and event migration.

Reference migration is matching clients, accounts, and keys with historical balances and limits; event migration is transferring raw events for the reconciliation period so that the new pipeline reproduces the same numbers. Until the amounts match, charges follow the old rules.

Phased implementation is structured as follows: first, shadow mode, where the new pipeline calculates in parallel and does not charge anything; then charging is switched to some accounts, while the rest continue through the old accounting.

At each step, there is a reconciliation report and a discrepancy threshold at which the step is rolled back. Full migration is possible only after discrepancies on live traffic are reduced to explainable ones.

This way, accounting stops being a black box: any amount can be broken down to the event, and a discrepancy can be shown to the client before they notice it.

Technical concerns: retries, limits, reconciliation

Five questions are heard most often when choosing a contractor for model consumption accounting. They are all about one thing: how pay-per-inference behaves when requests arrive in batches, fail by timeout, and diverge from what the client sees in the report.

How are limits and quotas set?

Restrictions are set at the level of key, client, and model — separately by number of calls and by volume of processed text over a time window.

On exhaustion, either a soft mode with a queue and clear response works, or a hard rejection; the rule changes without restarting the service.

How are charge refunds structured?

An erroneous charge is rolled back by a compensating operation rather than by deleting the record — a trail remains: who, when, and for what reason refunded it. A mass refund is processed as a batch over a selection for the period, with a log for each item.

How does amount reconciliation work and what to do with discrepancies?

Aggregates by client and model are regularly matched with primary events, and discrepancies automatically enter a report for investigation. The total in the report is always assembled from events, not from a precomputed total, so any line can be expanded down to a specific call.

Will accounting burden the main pipeline's load?

Events are written asynchronously, outside the critical path of the model response — an increase in the number of calls does not add generation latency. The buffer survives a service restart: during an update, no event is lost.

Let's discuss the task: accounting and billing scheme

For the task discussion to be based on facts rather than general considerations, we need access to the current accounting scheme: how a model request is recorded, at what point the billing event appears, and how it reaches the invoice.

Reviewing the current scheme usually reveals the main issues faster than subsequent discrepancy reconciliation — double accounting of retries, lost events on timeout, mismatch of units of measurement.

For a substantive review, it is enough to send:

• Request flow diagram: from the entry point to the model response, marked with exactly where the fact of processing is recorded.
• Log export for the period: request identifier, model, duration, status, retry flag — they show where the log diverges from invoices.
• Current billing logic: unit of accounting (request, tokens, runtime), rounding rules, handling of failures, timeouts, and partially completed requests.
• Description of billing integrations: how limits and prepayment are formed, where events go, who performs reconciliation and how.
• Reporting requirements: breakdowns by clients, models, and periods that must reconcile without manual adjustments.
• Pipeline constraints: peak load values, accounting latency requirements, data retention and anonymization policy.

After that, we will come back with an implementation plan, an estimate of the scope of work, and a breakdown by stages — from the accounting scheme and idempotent processing of pay-per-inference events to reconciliation and reporting.

If there is still little data, we will formulate a minimal set of checks on a limited scope: describe the task, and we will suggest where to start.