AI Agent Traffic

AI agent traffic is the set of website visits made by AI agents carrying out a task for a specific user, such as comparing products, booking a table or filling in a form. It is distinct from AI crawlers, which fetch pages in the background to build training sets or search indexes, and from AI-referred traffic, where a person clicks a link in an AI answer and arrives in their own browser.

Agent visits reach a site in three main forms:

• Cloud agents run on the vendor's infrastructure and usually identify themselves. Google added the Google-Agent user agent to its list of user-triggered fetchers on March 20, 2026, for agents hosted on Google infrastructure such as Project Mariner. It uses IP ranges from Google's user-triggered-agents.json, and Google is testing Web Bot Auth signatures under the agent.bot.goog identity. OpenAI's ChatGPT agent also signs its requests with Web Bot Auth.
• Agentic browsers and extensions run in the user's own browser: Perplexity Comet, Claude in Chrome, Gemini in Chrome auto browse and Browse with Copilot in Edge. Their requests usually look like normal Chrome traffic from a residential IP.
• User-triggered fetchers retrieve a single URL when a user pastes a link or asks about a page, for example ChatGPT-User, Claude-User and Perplexity-User. They sit between crawling and a full agent session.

One rule applies to all of them: because a user asked for the fetch, Google says its user-triggered fetchers generally ignore robots.txt, and the same holds for most agents acting on a user's behalf. Access decisions therefore move to the CDN, WAF or application.

How much agent traffic is there?

Volumes are growing from a small base, and every figure reflects one vendor's network rather than the whole web. Cloudflare CEO Matthew Prince said in June 2026 that automated traffic made up 57.3% of HTML page requests on Cloudflare's network, against 42.7% from humans, and pointed to fast growth in agent browsing. HUMAN Security's 2026 benchmark report said agentic traffic grew 7,851% year over year in 2025, and its July 2026 monthly data put Comet at about 47% of the agent traffic it observed, followed by Claude in Chrome at 24%.

Measuring and acting on it

Start by splitting log data into crawlers, user-triggered fetchers, declared or signed agents, and suspected unsigned agents. Verify claimed identities by IP range, reverse DNS or Web Bot Auth signature, never by the user-agent string alone, since Google warns it can be spoofed. AI crawler logs and crawler verification cover the declared part. Unsigned browser agents need behavioral signals, such as forms completed in a second or navigation bursts with no scrolling. Promptwatch's agent analytics identifies verified AI crawler and agent visits from CDN and server logs, including Google-Agent.

Then decide policy per agent and per path. Blocking a verified agent can turn away a real customer, while letting every data-center request through invites scrapers. Many teams allow signed agents on product, pricing and booking pages, rate-limit unknown automation, and keep normal authorization and confirmation steps for account changes and payments.

Finally, connect agent visits to outcomes. JavaScript analytics count some agent sessions as human visits and miss others entirely, which adds to AI dark traffic. Compare conversions on agent-heavy paths with server-side records to see whether agents are completing tasks or failing partway. Google's list of user-triggered fetchers documents Google-Agent's user agent strings and IP sources, and our Google-Agent bot page shows how to recognize it.