Bot Management for API Endpoints vs Web Browser Traffic

Bots made up 53% of global web traffic in 2025, and the "bad" kind alone accounted for 40% of that traffic, up from 37% the year before. Available reporting indicates this marks the seventh straight year bad-bot traffic has grown. Split the traffic into three buckets: good bots (13%, the search crawlers and uptime monitors doing legitimate work), bad bots (40%, the credential stuffers and scrapers and fraud tools), and a gray zone of AI crawlers and assistants that are legitimate by some definitions and a nuisance by others. Only 2.8% of websites had full protection against bot attacks in 2025, down from 8.4% a year earlier. The gap between how fast bots multiply and how well anyone defends against them is getting wider, not narrower, and treating a browser and an API endpoint as the same problem is a big part of why.

What makes API endpoints structurally different from browser surfaces as bot targets

Advanced bot traffic has a clear preference these days, and it's APIs. The Thales/Imperva API Threat Report for the first half of 2025, which looked across more than 4,000 environments worldwide, found that 44% of advanced bot traffic now targets API endpoints specifically. The reason isn't complicated: an API hands over application logic directly, with no rendering layer standing between a request and a response, no session state to fake, none of the friction a webpage throws up. Scraping a rendered page means downloading images, running scripts, waiting for a DOM to settle. Calling an API means sending a request and getting the data back, structured and ready to use. It's just less work.

The intent behind that traffic breaks down cleanly. Data scraping accounts for 31% of API bot activity, payment and coupon fraud for 26%, account takeover for 12%, and scalping or inventory manipulation for 11%. None of these are smash-and-grab operations. They're businesslogic attacks: a bot manipulating a pricing endpoint, chaining together authentication steps in a sequence a real user never would, or pulling data out in small enough increments that each individual request looks like normal usage. The Thales/Imperva report logged over 40,000 API security incidents in the first half of 2025 alone, putting the year on pace to clear 80,000. Financial services took the brunt of API-focused DDoS traffic, at 27%.

Separate numbers from Akamai's own tracking show daily API attacks up 113% year over year, and 61% of 2025's API attacks involved unauthorized workflows or activity patterns that didn't match how the API was supposed to be used. Thales/Imperva found that credential-stuffing and account-takeover attempts against APIs lacking adaptive multi-factor authentication rose 40%. And APIs tend to sit in a blind spot: there's no interface for a security analyst to eyeball, and plenty of organizations don't have a complete inventory of which endpoints exist and what they expose. A browser bot has to simulate an entire user session, complete with rendering and mouse movement and a plausible click path. An API bot skips all of that and just calls the endpoint. Leaner surface, faster attack, and much harder to tell from a legitimate integration partner making normal calls.

How browser-based bot detection works and where it breaks down

Browser bot detection leans on a stack of signals that only exist because a browser renders a page: whether JavaScript actually executes, whether a headless environment gives itself away, how a mouse moves across the screen, how long someone pauses before clicking, and dozens of small fingerprintable details about the device and browser itself. Add in session continuity (does the click path make sense) and IP reputation, and that's roughly the toolkit.

The trouble is that bad bots have gotten good at wearing a costume. Imperva's Bad Bot Report found that in 2025, 41% of bad-bot traffic identified itself as Chrome, up from 39% the year before, with Android Browser a distant second at 17%. Twenty-one percent of bot attacks route through residential proxies, IP addresses handed out by actual internet providers to actual households, which makes the traffic look like it's coming from someone's home network rather than a data center in another country. IP-reputation blocking alone doesn't stand a chance against that.

CAPTCHA, the old standby, has more or less stopped working as a serious control. Researchers at ETH Zurich showed in 2024 that a YOLO-based model beat Google's reCAPTCHAv2 with a 100% success rate after training on just 14,000 labeled images. Commercial CAPTCHA-solving services now clear challenges in under a second for a fraction of a cent. What's left is a control that annoys real customers while doing basically nothing to a motivated attacker. That's not a tradeoff, that's a bad deal.

Then there's the difficulty of naming names. A lot of the highest-volume "bad" traffic isn't anonymous or criminal at all, it's named and semi-legitimate. ByteSpider Bot, run by ByteDance, accounted for 54% of all AI-enabled attack traffic. Applebot made up 26%, Anthropic's ClaudeBot 13%, and the ChatGPT User Bot 6%. These crawlers aren't hiding, they're just scraping at a volume and pattern that crosses into abuse. Blocking them isn't a technical detection problem so much as a policy call about who gets to crawl a site and how much. DataDome's 2025 Global Bot Security Report found that 64% of AI bot traffic targeted web forms, with only 5% reaching checkout flows, meaning browser-facing bots cluster around data capture rather than transactions. Behavioral signals still work, but the ceiling is dropping: every "human" mouse-movement pattern gets fed into a training set eventually.

Why the detection signals available for browser traffic don't transfer to API endpoints

None of the browser toolkit works on an API call, and the reason is mechanical rather than philosophical. JavaScript fingerprinting requires a rendering engine to execute the script in the first place. A headless API request never renders anything, so there's no execution context to fingerprint, no DOM to interact with, no mouse to move. The entire client-side layer that browser detection depends on simply isn't present.

What's present instead is a different set of signals: authentication tokens and API keys (which get stolen and shared more often than anyone would like), the order and composition of request headers, the sequence and timing of endpoint calls, how sessions and tokens get reused or rotated, request volume against individual endpoints rather than the site as a whole, and oddities in parameter values or payload shapes.

The sharpest problem is that API bots frequently show up already authenticated. Credential stuffing succeeds often enough that a meaningful share of API bot traffic is carrying valid, working credentials, which guts any control that treats "authenticated" as a proxy for "legitimate." A bot hitting a pricing endpoint 10,000 times with a valid token doesn't trip a network-layer alarm. At the network layer, it looks exactly like a heavy integration partner doing its job.

F5 Labs' 2025 Advanced Persistent Bots Report analyzed over 200 billion web and API transactions between November 2023 and September 2024, and found average automated traffic is 10.2% post-mitigation across industries, but the profile of that traffic differed sharply between mobile API endpoints and web endpoints, with different seasonality and different sophistication. The entertainment industry saw 23% of its mobile API traffic come from unauthorized bots, while hospitality saw nearly 45% of its web platform transactions automated. Same underlying problem, completely different shape depending on the surface. Mitigation strategies do not carry over from one to the other. API defense has to run on behavioral baselines, sequence analysis, and rate controls built around what the business logic actually expects, because the client-side signals browser detection leans on simply aren't in the request.

Detection architecture for API endpoints: what effective controls look like

Start with a baseline. Effective API defense means knowing what normal traffic looks like for each endpoint, each class of credential, and each window of time, so that a deviation actually means something instead of getting lost in the noise.

Sequence matters more than any single request. Legitimate integrations call endpoints in predictable order, in a pattern that matches how the workflow is supposed to run. A bot exploiting business logic often skips a step a real client would hit, or calls things in an order that only makes sense if the goal is to extract data or manipulate a price rather than complete a transaction. Given that unauthorized workflows made up 61% of API attacks according to 2025 tracking, catching this requires watching the sequence of calls, not inspecting each request in isolation.

Authentication signals need the same scrutiny: token sharing, a token suddenly appearing from an unfamiliar IP range, login patterns that don't match how that client has behaved before. Catching credential stuffing at the API layer means correlating failed and successful attempts across every endpoint that accepts a login, not just watching one route.

Rate limiting has to get specific. A site-wide threshold is too blunt to catch a bot that hammers one pricing endpoint while staying comfortably under the aggregate limit. Controls need to work at the level of each endpoint, each token, and each IP, tied to what that endpoint's business logic should reasonably see. A policy question causes all of it: a machine can't answer on its own which crawlers and integrations the organization actually wants to allow, and that unresolved choice produces the trouble described above. That's a judgment call, not a switch. Web application firewall rules tuned to the shape of the API payload, expected schema, expected parameters, rejecting anything that doesn't match the contract, round out the layer.

Detection architecture for browser traffic: moving past fingerprinting and CAPTCHA toward intent signals

Browser defense gets to use signals APIs never will, but none of those signals hold up alone anymore. Each one, taken individually, is spoofable. The value comes from stacking them.

That stack includes client-side JavaScript telemetry (how the browser environment behaves, whether DOM interactions time out the way a real render would), behavioral biometrics like mouse trajectory and keystroke cadence and scroll physics (still hard to fake consistently at scale, even with AI in the loop), device fingerprinting across many attributes at once, where the combination and its stability over time matter more than any single data point, and session path coherence, meaning whether the sequence of pages visited tells a story a human would actually live out, rather than jumping straight to checkout with no browsing in between. IP reputation still has a role, layered with ASN and hosting-provider classification and specific detection for residential proxies, since 21% of bot attacks now route through residential proxies specifically to dodge reputation-based blocking.

CAPTCHA's replacement isn't another puzzle, it's something the user never sees unless risk actually warrants it: cryptographic proof-of-work or browser environment attestation, served only once a risk score crosses a set threshold rather than gating every visitor by default. Vendors selling this kind of adaptive, invisible challenge pair it with layered detection and checks against known-compromised credentials, rather than relying on any single gate.

Binary block-or-allow decisions are a blunt instrument. A better approach scores traffic on a continuous scale, say 1 to 99, and assigns graduated responses, allow, log, challenge, or block, at thresholds someone actually configures on purpose. That keeps a low-confidence signal from locking out a real customer. Good bots still need an allowlist: search crawlers, uptime monitors, sanctioned AI discovery agents shouldn't get treated as suspicious just because they're automated. Reframe the actual question being asked. It was always whether the traffic is doing something it's authorized to do." It's whether the traffic is doing something it's authorized to do. Wanted bots exist. Unwanted humans exist too. Detection has to weigh behavior and intent, not just what kind of client showed up.

Enforcement mechanisms and policy logic that differ by surface

Once detection flags something, the response needs to match the surface and the stakes, not follow one script for everything.

On browser traffic, an invisible challenge, proof-of-work or browser attestation running quietly in the background, works as the default for medium-confidence suspicious traffic, since a genuine visitor never notices it fired. Interactive challenges, the kind that ask a visitor to do something, should get reserved for the moments where the stakes justify the friction: a login attempt, a payment being initiated. Outright blocking belongs only at the high end of the confidence scale, because a false positive on browser traffic means a real customer got locked out of something they were entitled to use.

On the API side, enforcement runs on tighter, quieter dials: throttling a specific endpoint rather than blocking a client outright, revoking a token, or forcing re-authentication when a sequence of calls stops matching the pattern that endpoint normally sees. There's no interactive challenge to fall back on: an API caller can't solve a puzzle. That's exactly why the policy logic upstream, deciding in advance what a legitimate workflow looks like for that endpoint, carries more of the weight than the enforcement action itself. Get the baseline wrong, and no amount of rate-limiting downstream fixes it.