Why does AGI Agent visit websites?

It visits while carrying out a user-requested task, which may include navigating a site, using an app workflow, or making a purchase.

Does AGI Agent respect robots.txt?

The metadata does not claim robots.txt support, and Cloudflare explicitly records false. Use an application, CDN, or WAF rule when access must be restricted.

How can I verify AGI Agent is genuine?

Validate the request's HTTP message signature with the public keys at `https://api.agi.tech/.well-known/http-message-signatures-directory`. Do not trust a name in the User-Agent header.

Does a valid signature authorize a purchase?

No. The signature identifies the agent. Your site must still authenticate the customer, check permissions, and require whatever confirmation its purchase flow normally needs.

Does AGI Agent affect AI search or model training?

No such use is documented. AGI Agent performs live tasks for users rather than indexing pages for answers or collecting a training corpus.

AGI Agent: AI Crawler Guide & robots.txt

What is AGI Agent?

AGI Agent is a user-initiated productivity agent from AGI, Inc. It can take actions and make purchases for a person, so its traffic represents an attempted task rather than a background crawl for an index.

AGI describes AGI-0 as an agent that can operate apps and websites from phones, computers, and other devices. Its examples include ordering rides or food, making online purchases, editing media, and sending messages. A website may therefore see several related requests as the agent works through one workflow.

AGI's API documentation models browser work as a session that receives a task. That context matters when reviewing logs: page loads, form submissions, and follow-up navigation may all belong to the same user instruction. The site still decides whether the session is authenticated and whether a requested action is allowed.

This agent has no documented role in public AI search or model training. Allowing it can help an AGI user complete a task on your site, but it does not make your pages more likely to appear in generated answers. Blocking it affects automation of the live workflow instead.

Cloudflare's bot directory lists AGI Agent as an AGENT in its AI assistant category, names AGI, Inc as the operator, and links to the AGI-0 product page. It records no user-agent patterns or complete User-Agent strings. As a result, there is no stable text token for a narrow robots.txt group or log classifier.

Cloudflare does record a Web Bot Auth directory at https://api.agi.tech/.well-known/http-message-signatures-directory and marks followsRobotsTxt as false. A valid HTTP message signature checked against that directory can establish that a request came from AGI Agent. It cannot prove what the user intended, grant permission to transact, or replace the site's own authorization checks.

Relevant for AI searchAGI agent

Is AGI Agent relevant for AI search?

Yes. AGI Agent feeds AGI agent, so the pages it can reach shape what those AI products say about you.

AGI Agent crawls and indexes pages so the AI search or assistant behind it can retrieve them at answer time. A page it has never fetched cannot be quoted, summarized, or linked in that product's answers, so most sites keep it allowed to stay citable. Blocking it removes your pages from that AI surface and hands those citations to competitors.

How to handle AGI Agent

Decide first whether your site permits automated agents to browse or transact for users. If it does, apply the usual login, inventory, payment, confirmation, rate, and abuse controls. A verified agent should receive no broader authority than the person or account behind the task.

Do not publish a bot-specific robots.txt snippet for AGI Agent. No stable user-agent token is recorded, and Cloudflare says the agent does not follow robots.txt. A display-name rule would be both easy to spoof and unlikely to provide dependable enforcement.

For identity-based policy, validate the HTTP message signature against https://api.agi.tech/.well-known/http-message-signatures-directory. Reject an invalid signature when your rule requires verified AGI traffic. You may still block a correctly signed request based on the requested path or action.

Watch a complete session before judging isolated requests. A failed form submission may be a compatibility problem, while repeated unauthorized attempts or access to disallowed functions calls for an application or edge block.

Examples

A retailer accepts a correctly signed AGI Agent session, but checkout still requires the shopper's normal payment confirmation.
A financial service blocks browser automation on account-transfer routes even when AGI Agent presents a valid identity signature.
An edge rule rejects a request claiming to be AGI Agent because its HTTP message signature cannot be verified against AGI's directory.

Frequently asked questions about AGI Agent

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

AGI, Inc operates AGI Agent. Cloudflare's operator link points to the company's AGI-0 product page.

Related bots

Amazon Kendra

Amazon Kendra is a managed information retrieval and intelligent search service that uses natural language processing and advanced deep learning model.

AI Assistant