Controlling AI agent costs: leak points and causes
Tracking model calls is almost always calibrated to average load: the agent responds to single requests, and the counter matches reality.
A peak spike breaks this picture — parallel dialog branches and retries after a timeout consume the default reserve in minutes instead of the allotted period.
Industry incident reviews agree that up to a third of overspend in agent systems comes from unmanaged peaks, and another roughly a quarter from duplicate requests.
The agent re-queries the model on the same context, retries are not filtered out, and a significant portion of calls goes to waste without adding anything to the response.
The third leak point is integration failures. An external service returns an error, the logic enters a retry loop, and each individual request in the log looks normal: there is no anomalous call, while the total load grows unnoticed. Default limits do not help here — they are either not set or set at a level that never triggers.
Therefore, it makes sense to tie limits not to the overall budget but to the load profile. Paperclip provides the constraint mechanism, but the rule set for a specific agent is still designed around its scenarios.
We'll break down how this is addressed: from call accounting to thresholds that trigger before a spike, not after.
What limit and call rule management provides
Call rules are the boundaries within which an agent operates predictably: a limit on the number of concurrent requests, separate ceilings for expensive operations, and a priority queue by domain. Below are two effects visible in metrics rather than in feelings.
Cutting peak spikes
Limiting concurrent calls per agent and per scenario prevents one long dialog from occupying the entire available bandwidth. A spike that previously spread across the system and increased response time for others now hits the ceiling and is handled by the queue.
Queue priorities determine who gets through first: short requests go immediately, while large analyses wait for a window.
The share of rejected calls becomes a manageable value — with well-calibrated settings it stays within a few percent and predictably decreases after quota expansion, while load remains predictable during peak hours.
Reducing incident investigation time
In Paperclip, every rejected call is logged with a reason: which limit triggered, on which agent and scenario, and at what time. Previously this reason dissolved into the general event stream, and investigation took hours of correlating scattered records.
Now an incident has a name, and investigation time is reduced severalfold — from the moment of the signal to a clear decision.
A whole class of failures disappears entirely: those that arose from agents overheating each other are no longer reproduced because the system never reaches the state in which they appeared.
Ways to limit calls: comparing approaches
A call count limit is set in two places: where the agent forms the request and where the model receives it. The first option is closer for those who own the agent code; the second is for those managing several teams and wanting one rule for everyone. The launch platform, including Paperclip, usually already counts calls per run.
Hard limits vs. adaptive quotas
A hard limit is a counter that returns a rejection when exhausted: behavior is predictable, debugging is simple, but a peak run is cut off midway.
An adaptive quota distributes the remainder across the window and reduces the call rate instead of aborting the task; the cost is increased latency, which must be shown separately in monitoring.
| Option | When it fits | Limitations |
|---|---|---|
| Hard limit on number of calls | rare single runs | cutoff at peak, manual restart |
| Adaptive quota | stream of user requests | more complex configuration, response time increases |
| Platform-side limit | multiple services, single rule | does not see agent logic, rejection arrives late |
| Service-side limit | need your own degradation scenario | duplicated in each service, diverges on edits |
Platform-side and service-side limits
A platform-side rule is the same for everyone and cannot be bypassed from code, but the agent only learns about exhaustion from a rejection.
Inside the service, context is visible — step, branch, remaining quota — so soft thresholds are set there, while the hard stop remains external. A working scheme: an external limit as insurance plus an internal counter for managed degradation.
Stages of configuring limits and quotas in your environment
As long as limits live in one engineer's head, any edit is done blindly: it is unclear which agent consumes how much and what the new threshold will cut off.
We go in reverse order — first a call map and a constraint scheme, then configuration, testing on a staging environment, and handover of procedures to your team. We do not stop working agents during the work.
- Call inventory. We gather which agents, scenarios, and background tasks call the models, at what frequency, and with what context volume. The output is a call registry with owner and priority: without it, a limit will cut what is needed and let excess through.
- Limit scheme. We define levels — scenario, agent, department — and types of constraints: daily volume, request frequency, depth of reasoning chain. Here we also fix behavior when quota is exhausted: switch to an economical model, queue, or stop.
- Rule configuration. We transfer the scheme into configuration: access keys, counters, averaging windows, warning thresholds. Rules are versioned together with code, so a limit change goes through review rather than being edited in a panel "by eye."
- Staging validation before handover. We run scenarios with deliberately lowered thresholds and confirm that exactly the intended behavior triggers: the agent switches to fallback mode rather than failing with an error. We separately test rollback and behavior under load spikes.
- Handover of procedures. We provide the call map, limit scheme, procedure for changing thresholds, and incident investigation. Your engineers then change quotas themselves, without contacting us.
Deliverables: configurations, schemes, and procedures
We hand over not a description of "how it should be done" but a working kit: an operations engineer applies it without us and without reading through correspondence.
Everything concerning agent costs is provided as configurations, schemes, and procedures with explicit owners.
Schemes and failure rules
- Limit configurations — limit per task, dialog, and day, warning threshold and action when reached: soft stop with a prompt, hard stop, or switch to economical mode.
- Routing schemes — which task is handled by which tool, where caching is enabled, and at what point a human is involved.
- Access policies — who can raise a limit, who can only view spend, which changes require a second approval.
- Logs and alerts — spend by agent, task, and user, notifications a few percent before the limit, daily summary to responsible parties.
- Failure rules — behavior when the limit is exhausted, rollback procedure for the latest configuration, which circuit remains operational.
- Support instructions — typical requests, where to look for the stop reason, what must not be changed manually, and whom to approach for a decision.
The documentation describes the current circuit rather than an abstract scheme: on the next change you edit one file instead of gathering knowledge from people.
Case study: how peak spikes stopped breaking the service
What it was like before
A service on Paperclip ran several agents, and each decided on its own how many model calls it needed. During peak hours, one agent consumed the shared limit in minutes: queues grew, other agents received rejections and went into retries, which finished off the load.
Recovery time was 40–60 minutes. The on-call engineer manually stopped the offending agent, waited for the queue to drain, and only then brought the service back up.
The culprit was found through indirect logs, so the failure class "shared limit exhaustion" recurred every week and took some user scenarios with it.
What it became after
Budget limits were moved to the level of an individual agent: daily and per-minute ceiling, share of the shared reserve, priority, and behavior on exhaustion — pause, switch to a lightweight model, or rejection.
Calls over the limit are rejected on entry, before reaching the model, so one agent's spike physically cannot occupy another's reserve.
Recovery time dropped to minutes: instead of manual intervention, rate limiting and automatic window reset trigger. The share of rejected calls stays at 1–3% and falls on background tasks rather than user ones.
The failure class "shared limit exhaustion" disappeared from the logs — not a single availability incident in a quarter.
What to do if limits are already set manually?
Manual limits in the Paperclip interface are a workable solution while there are five agents. The problem starts at dozens: the limit lives in a specific agent's card and is not inherited, and when adding a new performer you must remember to set it. There is also no way to check who changed a value and when — only the latest state remains.
We transfer the settings without stopping operations. First, inventory: we export current limits for all agents and consolidate them into a single list where discrepancies are visible — similar tasks have limits that differ severalfold due to manual edits.
On this basis, we describe rules: a shared limit per domain, refinement at the team level, and a targeted override for an individual agent. Inheritance provides one place for changes instead of a dozen cards.
Next is a gradual transition. We enable the new rules in observation mode: the scheme calculates which limit would apply and compares it with what is set manually. We analyze discrepancies while nothing is blocked, so agent operations are not interrupted.
Then we enable enforcement — first on one group of agents, then on the rest, with the ability to revert to the previous value preserved.
After the transition, manual limits in the interface remain only as deliberate exceptions visible in the common list. Everything else is calculated by the scheme: a new agent receives limits through inheritance, and changing a common rule does not require going through cards.
Technical doubts about limits and logs
Limits work in tandem with the call log: a threshold without a record is useless — there is nothing to investigate the trigger with.
Therefore, during configuration we immediately fix what is saved for each agent request: initiator, operation type, spend, and whether it hit the limit.
The questions below are those that most often come up during acceptance: they concern not only quota numbers but also system behavior in contentious situations. We separately cover how limits are revised when the team or load grows.
Will the limit trigger falsely?
False triggers come from a single rough threshold applied to all requests indiscriminately. We separate limits by operation type: short service calls get their own higher threshold, expensive ones get their own. We add a smoothing window: a one-off agent spike is not blocked; a sustained excess is blocked.
Will the response be cut off midway?
The limit is set for the entire task processing cycle, not for an individual call. The agent either fits within the frame and finishes the response, or does not start it.
A warning about approaching the boundary goes to the agent owner before the stop — there are no cut-offs mid-word in a user dialog.
Can different limits be set for teams and departments?
Yes, a quota is tied to the hierarchy: department — team — specific agent. Inheritance goes top-down with the right to override at the lower level. Support and analytics have different request intensity, so a common threshold for them is definitely wrong.
What exactly is written to the call log?
Each request leaves a record: who initiated it, which tool was called, how much was spent, and whether it stayed within the limit. This is enough to reconstruct the chain for an incident and distinguish a working spike from an agent loop.
How are quotas revised?
Revision is a scheduled procedure: we review the log for the period, the share of rejections by limit, and the dynamics of the number of agents, then raise or lower the threshold at the required hierarchy level. The change is applied without restarting agents, and old values are saved in history.
Let's discuss the task: we'll assemble a limit scheme for your environment
To ensure the limit scheme does not break working scenarios, we need input on your environment: which agents are launched, who calls them and how often, where manual launch ends and automatic begins, and who has authority to change a limit.
We separately review current settings — they show where spend has already gone out of control and where the reserve is excessive and cuts useful scenarios.
We gather this data and return with an estimate of timelines, an implementation plan, and a list of steps. Below is what we take from your side and what we give in return.
- Inventory of agents and scenarios: for each — owner, criticality, and allowable spend ceiling per period.
- Review of current settings: what is already set, where thresholds trigger, and where calls bypass common constraints.
- Responsibility boundaries: who approves a limit, who receives a notification when it is reached, and how a one-time increase is processed.
- Three-level limit scheme — per individual scenario, per team, and per common circuit, with priority rules between them.
- Behavior when the ceiling is reached: warning, switch to a more economical mode, or full stop — separately for each scenario.
- Phased implementation plan with checkpoints: first observation and statistics collection, then constraints in warning mode, and only then hard limits.
- Documentation kit: limit scheme, procedure for changing limits, actions on trigger, and rollback to previous settings.
Describe the task and show current settings — we'll return with an estimate and plan.
