Large enterprises running inference workloads at scale report a consistent problem: estimated tokens do not match billed tokens. The variance often ranges from 5% to 15%, depending on the provider, the caching strategy, and prompt complexity.
At low volumes, this variance is a rounding error. At scale, it is a budget line item.
Practical Outcomes
- Finance can reconcile AI invoices to internal usage logs instead of carrying unexplained variance.
- Teams can attribute spend to products, workloads, and policy changes with audit-ready records.
- Procurement can negotiate and dispute charges using evidence instead of estimates.
- Leadership gets more accurate budget forecasts for AI operating costs.
The Financial Weight of a Technical Gap
A 10% variance on a $2 million monthly inference bill is $200,000. Over a fiscal year, that is $2.4 million in unexplained cost. Finance teams cannot reconcile this against internal usage logs. Audit trails do not close. Budget forecasts drift from actuals in ways that cannot be attributed to specific teams, products, or workloads.
This is a procurement and audit problem more than a research problem. The tokens were consumed. The invoice arrived. The internal accounting does not match the external charge.
Many organizations treat AI inference as a cost center they can monitor but not fully control. That approach breaks down as spend moves from experimental budgets into operating expense.
Caching Increases Opacity
Caching is often presented as a cost optimization. In practice, it introduces accounting complexity that few organizations are prepared to handle.
When a response is served from cache, the provider may bill zero tokens, partial tokens, or full tokens depending on the caching policy and billing model. The user cannot verify which case applied. The cache hit rate is not reported at the granularity needed for reconciliation. The savings are estimated, not measured.
Worse, caching breaks attribution. A request that would have consumed 4,000 tokens on Tuesday consumes 200 tokens on Wednesday because a similar request populated the cache. The internal system logged the same input. The invoice reflects different consumption. The delta cannot be traced to a specific decision or policy.
Finance teams are asked to trust that caching reduced costs. They cannot verify it. They cannot audit it. They cannot explain it to a regulator or an internal review board.
Attribution and Reconciliation Fail Together
Token billing disputes arise when three records do not agree: the internal usage log, the provider's metering, and the invoice.
Internal logs capture what was sent. Provider metering captures what was processed. The invoice captures what was charged. These three numbers should match. In practice, they diverge.
The divergence has multiple causes. Prompt preprocessing by the provider may add or remove tokens. System prompts may be metered differently than user prompts. Caching may reduce metered tokens without reducing logged tokens. Retry logic may double-count failed requests. Batching may aggregate requests in ways that obscure per-request attribution.
Most of these causes are implementation details that were not designed for auditability. The result is token billing that cannot be audited at the level finance and compliance teams need.
Verifiable Token Accounting
The corrective mechanism is verifiable token accounting: a system where every token consumed can be traced from request to response to invoice.
This requires three properties.
Determinism. The same input must produce the same token count every time. If a prompt is tokenized differently on different runs, reconciliation is impossible. The tokenization must be stable, versioned, and reproducible.
Replayability. Any request must be replayable against the billing record. If the organization disputes a charge, they must be able to reconstruct the exact input, tokenization, and output that generated that charge. Without replayability, disputes are assertions without evidence.
Auditability. The full chain from request to billing must be inspectable. This includes the raw input, the tokenized form, the model routing, the output, and the metering event. Each step must produce a record that can be independently verified.
These are baseline requirements for treating inference as a governed cost center.
The Control Surface
Token accounting is a financial control surface. It determines what can be measured, what can be attributed, and what can be audited.
Organizations that lack verifiable token accounting operate with a gap between their internal view of AI costs and their external obligations. That gap is measured in dollars, and it compounds with scale.
Closing the gap requires infrastructure that was designed for accountability. The tokens must be countable. The counts must be reproducible. The records must be auditable.
Call it what it is: a financial control.