Why Per-Second Billing Breaks Down for Multi-Minute AI Jobs
Agents spend most of their time waiting on I/O, but serverless billing charges for every second.
The serverless billing model was not designed for AI agents. Per-second billing emerged from a specific set of assumptions about what workloads look like, and those assumptions are violated, systematically, by every multi-minute agent job running in production today. The overcharges are not incidental. They are structural.
To understand why the billing breaks, you have to understand what the agent is actually doing with its time. An agent is not a function. It is a loop: plan, call a tool, receive a result, update context, plan again. Each iteration is a round trip to somewhere external, usually an LLM, sometimes a search index, a database, a code execution environment, or a human who needs to approve the next step.
What that looks like from inside the execution timeline is mostly waiting. The agent serializes its full conversation history, sends it to the model, and sits idle while inference runs. That inference takes seconds per call, sometimes longer for reasoning-class models that chain their own internal steps. When the result comes back, the agent processes it briefly, decides what to do next, and waits again. Tool calls introduce another layer of latency. Human-in-the-loop approvals introduce pauses measured in minutes.
The CPU is not working during any of this. It is blocked on I/O.
State must survive across all of it, which is the second structural mismatch. The agent's filesystem, memory, and intermediate results need to persist from one step to the next. Classical serverless invocations are stateless by design; each cold start arrives with no memory of what came before. That design choice was sensible for a webhook handler. For an agent 12 steps into a research task, it is a liability that compounds with every subsequent invocation.
The scale makes the mismatch concrete. JPMorgan Chase has reported running more than 450 AI agent use cases, with tasks that previously required hours of analyst time completing in significantly less time. Those completions are dominated by inference latency and data retrieval, not CPU execution. Per research cited by CockroachLabs, agentic workloads consume between 5 and 30 times more tokens per task than a standard chatbot query. The I/O surface area is orders of magnitude larger than anything serverless was designed to accommodate, and the billing model has not caught up.
The three specific places wall-clock billing overcharges agent workloads
Billing for idle I/O wait. Every second an agent spends waiting for an LLM response, a tool result, or a human approval is billed at the same rate as active compute. Duration is duration. The billing system does not know whether your code was executing logic or staring at a socket, and it charges the same either way. If inference runs slow on a given request, your bill goes up proportionally, even though your code did nothing additional. Cloudflare, an internet infrastructure and DDoS-protection provider that runs serverless compute at the edge, has published analysis of Workers workloads finding that most applications spend far more time waiting on I/O than using the CPU; for agent jobs, that ratio is more extreme still.
The redundant-context tax. Agents don't send just the new prompt at each step. They send the full conversation history every time, because the model has no persistent memory and must reconstruct context from scratch on each call. By step 20 of a long research task, the agent has transmitted the same early-conversation tokens 20 times. CockroachLabs-cited research puts re-sent context at 62% of total agent inference bills. Wall-clock billing has no mechanism to distinguish tokens the model needed for the first time from tokens it had already processed; it charges for the full inference wait regardless of how much of that wait was redundant. This cost is rarely advertised. You find out on the invoice.
Execution timeouts as a hard structural wall. Classical serverless platforms impose maximum execution duration limits, a constraint designed for millisecond functions. It is a hard wall for workflows that legitimately run for minutes, or that require a human approval to proceed. When an agent hits the timeout, state is lost, the job must restart, and the failed attempt is still billed. AWS's introduction of extended-duration capabilities in its serverless offerings is an acknowledgment that the original model could not handle these workloads. Cold starts compound the problem: when bursty agent traffic triggers new instance provisioning, spin-up latency is charged before the agent does a single useful thing.
What runaway costs look like when these failures compound in production
The gap between prototype cost and production cost for AI agent workloads is not incremental. It is a step function, and most organizations discover this after the fact, usually while explaining the invoice to someone who approved a much smaller budget. Infrastructure overhead, reliability engineering, monitoring, and retry logic add costs that prototype benchmarks do not capture; per Hypersense Software's analysis, most enterprise budgets underestimate the true total cost of ownership by 40 to 60%.
The more important distinction is between monitoring and enforcement. Knowing a cost is accumulating is categorically different from having a mechanism that stops it. Wall-clock billing with alert-only guardrails is exposed to runaway cost scenarios because the billing unit, elapsed time, has no natural ceiling that the workload itself can enforce. Alerts tell you the barn is on fire. They do not close the barn door.
At enterprise scale, the numbers are instructive. Uber's CTO has publicly reported the company exhausting its annual AI coding budget well ahead of schedule, with monthly API costs per engineer running between $500 and $2,000. These are not edge cases caused by careless engineers. They are predictable outputs of applying a millisecond-oriented billing model to multi-minute workloads without a spend enforcement layer. The model is doing exactly what it was designed to do; the problem is that it was designed for something else entirely.
Why the gap between wall-clock time and CPU time is the right place to look for a fix
The root cause across all three failure modes is billing for time elapsed rather than compute consumed. Everything else is a symptom.
CPU time measures only what the code actually did: processing inputs, executing logic, writing outputs. It is unaffected by how long an upstream service took to respond. For a workload that spends the majority of its duration waiting on LLM inference, tool APIs, or human approvals, the delta between CPU time and wall-clock time is substantial, and that delta is precisely the amount the organization is being overcharged.
Cloudflare Workers, part of Cloudflare's global edge and security network, prices on CPU time rather than duration. The practical consequence is that idle I/O wait is simply not billed. At tens of millions of monthly API requests averaging 5ms of CPU time, the Workers Paid total is substantially lower than equivalent volume through duration-based platforms. The difference is almost entirely I/O wait that one platform charges for and the other does not. At prototype scale, that gap is easy to miss. At production scale, it becomes the central line item in an uncomfortable conversation.
CPU-time billing does not, on its own, solve the redundant-context problem or the timeout problem. Those require additional architectural responses. But it removes the most structurally unjust component of duration-based billing for I/O-bound workloads, which is a reasonable place to start.
What a billing model that fits multi-minute AI jobs actually requires
CPU-time pricing is the starting point, not the finish line. A billing model actually designed for agent workloads requires several additional properties, and it is worth being specific about what those are, because vague calls for "better tooling" produce better dashboards rather than better bills.
Charge for active compute, not elapsed time, and apply that principle across the full stack, including GPU inference. Cloudflare Workers AI, an AI inference service running across Cloudflare's global edge network, prices based on actual GPU compute consumed per request. The unit of billing is work performed, which is the right abstraction and also, not coincidentally, the one most favorable to workloads that spend most of their time doing nothing.
No penalty for legitimate pauses. Human-in-the-loop approvals, tool latency, and multi-session state persistence should not generate billing events. An agent waiting for a human to approve a document retrieval step is consuming no resources; charging for that wait is charging for time passing, nothing more.
Spend enforcement, not just monitoring. Configurable hard ceilings per invocation or per workflow are a requirement. Alerts are observability; ceilings are control. These are not the same instrument, and the organizations that conflate them are the ones publishing retrospective case studies about unexpected cost overruns.
Transparent token-level accounting. Organizations need visibility into how much of their inference bill is new context versus re-sent context. Without that breakdown, the redundant-context tax remains invisible, and invisible costs persist precisely because nobody is accountable for them.
Scale-to-zero as a default. Organizations should not pay for warm instances between agent jobs. This should be baseline behavior, not a premium feature.
The MCP standard, now under Linux Foundation governance, matters here for a less obvious reason. Standardized tool connectivity creates a clean, auditable boundary between agent compute and tool I/O, which is a prerequisite for billing systems that can actually distinguish active compute from wait time. Without a standard interface, that boundary cannot be defined consistently across platforms, and the billing problem becomes a platform-specific negotiation rather than a solvable engineering question.
The broader shift in AI pricing, from flat subscriptions toward metered models based on actual processing, is correct in direction. The question is whether the meter measures the right thing. Wall-clock time was a defensible proxy in 2014, when functions were short-lived and the gap between CPU time and elapsed time was small enough to round away. That gap is no longer small. For multi-minute agent workloads, it is the cost structure, and the sooner billing models reflect that, the fewer budget post-mortems the industry will need to conduct.
