AI-Generated Phishing Detection at Email Gateway Scale

AI-generated phishing detection at email gateway scale requires a change in what defenses read. Messages that read well and are written for one person remove the flaws that older filters used as signs of fraud. Detection needs to connect sender behavior, infrastructure provenance, and campaign patterns, with governed AI agents helping that detection keep pace.

How email gateways were built to read signals that no longer exist

Legacy Secure Email Gateways were engineered for high-volume, low-craft phishing. Their detection stacks matched message signatures, checked domain reputations, compared URLs and IPs against blocklists, and weighted clues such as spelling errors and generic greetings. Attackers reused templates and sending infrastructure because it took skill to write messages that felt real and were tailored to each person.

IBM X-Force research published in 2023 put the effort required for a convincing, personalized spear-phishing email at roughly 16 hours. That constraint shaped the threat environment gateways were built for. An attacker could invest in a carefully researched lure or sacrifice quality to reach more people. Most campaigns chose volume and left fingerprints in their prose, templates, and infrastructure. Gateway design reflected that tradeoff rather than a failure to anticipate every possible deception.

What generative AI removed from the attacker's workflow — and from the defender's signal set

Generative AI removes much of the work that forced attackers to make those compromises. Fluent, native-quality prose takes grammatical errors out of the defender’s signal set, while recipient-specific generation reduces the need to reuse recognizable templates. Signature matching then has to check messages that are built differently even when they aim to carry out the same fraud.

Personalization has changed for the same reason. An AI-driven open-source intelligence pipeline can use LinkedIn, company sites, and social platforms to assemble references to a real manager, project, or vendor relationship. Research and drafting that once took long stretches of a person’s time can happen in minutes. A generic greeting is no longer the necessary price of scale. Nor does a believable reference to something happening at work prove that the sender belongs in the conversation.

A lure can read well, use wording found nowhere else, and be written for its recipient, so checks for poor writing or repeated wording have little to find. A correctly configured filter can lose its evidentiary basis without anything in its configuration changing. The spellchecker has not failed. It has simply been assigned a fraud investigation.

Where gateway architecture creates structural blind spots independent of message content

A perimeter gateway sees a message at delivery. MITRE ATT&CK classifies Resource Development as a tactic that includes the attacker’s preparatory activity. Lookalike domain registration, DNS setup, and sending infrastructure can precede a campaign by weeks. A delivery-stage inspection does not, by itself, expose that earlier trail.

The gateway’s position also limits which conversations it sees. A compromised account can send lateral phishing to colleagues within the same Microsoft 365 or Google Workspace environment without crossing the perimeter gateway. Authentication can make the situation more deceptive. A hijacked mailbox uses the legitimate domain’s signing keys, so domain reputation and DMARC can look clean while an attacker controls the account. Authenticating the domain does not authenticate the sender’s intentions.

Other attacks leave email after the first contact. Voice phishing, Teams impersonation, and SMS do not pass through email security, so a control at the inbox layer cannot check those exchanges. Better analysis of the messages visible to a gateway cannot reveal traffic that never passes through it. Detection therefore needs context from before delivery and coverage beyond the perimeter, rather than a more elaborate judgment about the arriving text alone.

Why shifting detection toward sender behavior and infrastructure provenance catches what content inspection misses

Infrastructure provenance supplies evidence that better prose cannot erase. DNS registration patterns, certificate data, and lookalike domain registrations reveal how sending infrastructure was assembled. That context exists outside the message itself. The trail of when and how that infrastructure was built can link a well-written lure to suspicious preparation even when its wording has no familiar signature. It also shifts attention toward attacker infrastructure and source disruption rather than treating each delivered email as an isolated event.

Sender behavior provides a different kind of context. Communication baselines can show when a supplier suddenly asks to change banking details or an internal account sends mail at unusual hours. Neither deviation depends on bad grammar. Graph analysis builds on this by mapping the links between senders, recipients, and domains, which helps reveal lookalikes and thread hijacking that a domain reputation lookup alone can miss.

Campaign correlation then links evidence from messages that each have their own wording. Organization-specific detection can use the organization’s vendor relationships and communication patterns. Content still has a role here. Natural language understanding can examine semantic intent, including urgency, authority claims, and unusual payment instructions, while computer vision decodes QR codes and renders landing pages that text inspection misses. The distinction is between reading what a message asks someone to do and judging whether its author remembered the apostrophes.

How agentic detection workflows close the gap that static vendor update cycles leave open

Attackers generating novel lures in minutes create a tempo problem for organizations waiting on vendor signature updates. Sublime Security describes autonomous detection engineering that generates new coverage from edge cases, tests it against historical email, and deploys it in hours. Historical testing supplies a check before deployment.

Agents can investigate user reports and system-flagged queues in seconds, reducing manual review from hours to minutes. Human review of every message becomes untenable at gateway scale. New hires, M&A targets, and small organizations may have too little email history for meaningful behavioral baselines. Infrastructure provenance and attacker-side graph analysis provide a complementary view that does not depend on a long organizational history.

The investigating agent also becomes an attack target. A crafted email can contain prompt-injection instructions designed to override its system prompt, so production detection needs input sanitization that identifies and neutralizes those attempts. Tool supply chains create another route. Postmark’s account of the MCP incident describes a counterfeit npm package impersonating its email server library that later secretly sent copies of messages to an attacker-controlled destination. That was not simply another suspicious email. It showed how a tool trusted by an agent could turn protective infrastructure into a delivery channel.

What the MCP stateless specification and edge inference change about deploying detection at scale

The July 2026 Model Context Protocol specification removes transport-level session management in favor of a stateless core. The protocol can therefore scale on ordinary HTTP-load-balanced infrastructure. Per-message classification is a natural fit for that model. Detection agents can run as functions around the world that each process inbound messages on their own, without keeping transport infrastructure running for each session.

Stateless processing does not mean detection has no history. Behavioral scoring still depends on communication baselines, and campaign correlation still depends on relationships across messages. The protocol change lets messages be processed on their own without each connection having to keep its own persistent session. It does not make an isolated email sufficient evidence for a campaign judgment.

At sustained high query volumes, it can offer lower per-unit costs, making per-message classification more practical. Bursty workloads can favor cloud inference, so edge deployment is not automatically the cheaper choice. Together, stateless routing and suitable edge inference give semantic analysis, behavioral scoring, and graph analysis deployment options beyond centralized persistent infrastructure, while leaving workload economics as a real constraint.

Why governance and identity controls on AI agent sessions are not optional additions to detection architecture

More deployable agents also mean more activity to govern. An agent that can inspect data, invoke tools, and consume inference resources introduces risks separate from the phishing it investigates. Microsoft’s internal MCP governance framework recommends placing every remote MCP server behind an API gateway. That provides one point for authentication, authorization, rate-limiting, and logging, with no direct calls around it.

Session controls also need to cover resource use. Each MCP-enabled agent session should have a declared token budget and cost limit, with overruns halted pending human authorization. Zero Trust adds continuous scrutiny after authentication, allowing the platform to terminate activity or require stronger authorization when an agent queries unusual data schemas or moves outside its declared scope.

Automation on the sending side needs a similar distinction. CRM integrations and marketing platforms send legitimate automated email, and agents that attackers control can also send large amounts of mail. Bot management and WAF rules need to distinguish those roles. The practical implication is broader than detection engineering. Without infrastructure governance, an organization can close a phishing gap while giving attackers new ways in through its agents’ permissions, tools, and access to resources.

What a detection architecture that does not depend on message content alone actually looks like in production

In production, these responsibilities form layers. Before delivery, infrastructure provenance and domain graph monitoring examine lookalike registrations, certificate issuance patterns, and DNS configuration anomalies on newly established sending infrastructure. At delivery, behavioral baselines and sender DNA score departures from established communication patterns, while semantic analysis examines authority, urgency, and unusual financial instructions. Computer vision brings QR codes and rendered landing pages into the same inspection picture.

Organization-specific detection links messages that use different words but share a suspicious pattern, while graph analysis maps the links between senders, recipients, and domains over their communication history. This layer assesses clean-looking messages in the context of their relationships, complementing the limited view of perimeter inspection. It does not pretend that an email gateway can suddenly inspect a voice call.

The agentic response layer turns findings into investigation and new coverage. Autonomous triage handles reports and flagged queues, while detection engineering tests new rules against historical email before closing exposed gaps. Input sanitization guards the agent’s interpretation of hostile material. Around that work, identity-aware session management, API gateway enforcement, token budgets, and continuous behavioral monitoring make the detection infrastructure governed and auditable.

Evaluation has a supporting role in this architecture. The NEST-Phish dataset, released in September 2026, provides matched synthetic legitimate and phishing emails with interpretable phishing-cue annotations for behavioral and machine-learning research. The synthetic provenance of its legitimate examples keeps benchmark evidence separate from production history, while allowing research into how people and classifiers respond to phishing cues. The assembled system ultimately rests on evidence an attacker cannot remove merely by improving a sentence, including how infrastructure was built, how an account behaves, and how messages connect.