Does TwinAgent affect AI search results or model training?

There is no documented search-indexing or model-training role for TwinAgent. Its visits represent browser automation for a workflow, so blocking it can interrupt that workflow but does not opt a site out of AI search or training.

Will TwinAgent obey robots.txt?

That behavior is not confirmed in this bot, and Cloudflare's bot directory marks TwinAgent as not following robots.txt. Publish a rule if useful, but enforce a required block at your edge or application.

Can TwinAgent reach pages behind a login?

Twin's Web Agent pauses at a login page so the user can sign in and complete any two-factor step. It then resumes with that authenticated session, and Twin says it stores the session cookies for later runs.

TwinAgent: AI Crawler Guide & robots.txt

Q: How do I verify a TwinAgent request?

Check a Twin browser workflow's Web Bot Auth signature against https://signature-agents.twin.so/.well-known/http-message-signatures-directory, which publishes the relevant public keys. The `TwinAgent` user-agent token alone can be spoofed.

What is TwinAgent?

TwinAgent is a network identity associated with Twin, a platform for building agents that carry out business workflows across APIs and websites. Twin's documentation says its Web Agent takes over when an API is unavailable or does not expose the action required for a task.

A Twin Web Agent run happens on an isolated cloud computer, separate from the user's local browser. The agent can inspect a page, navigate a dynamic interface, click controls, fill in forms, and extract information. During the build phase, Twin shows the planned browser task and asks the user to approve it. The user can then watch the run or cancel it.

For a site that requires authentication, Twin pauses at the login page and gives control to the user. After the user signs in, the agent resumes and Twin stores the session cookies for later runs. When TwinAgent is used for this browser workflow, its request can be part of an authenticated action rather than a fetch of a public page.

Twin does not describe TwinAgent as a search indexer or a model-training crawler. Allowing it does not make a page eligible for an AI search citation, and blocking it is not a training-data opt-out. The practical consequence is narrower: a Twin workflow that needs to use the site may fail.

TwinAgent has the stable token TwinAgent. Cloudflare's record lists TwinAgent/1.0 as the full user agent and identifies Twin as the operator. A user-agent string can be copied, so it is useful for log classification but not sufficient proof of identity.

Cloudflare also lists a Web Bot Auth key directory at https://signature-agents.twin.so/.well-known/http-message-signatures-directory. Signed requests can be checked against those keys. The robots.txt status in this bot is unknown, while Cloudflare's bot directory marks the agent as not following robots.txt, so a robots rule should not be your only enforcement point.

Relevant for AI search

Is TwinAgent relevant for AI search?

Yes. TwinAgent collects pages for an AI product, so what it can crawl influences how AI systems describe your brand.

TwinAgent crawls and indexes pages so the AI search or assistant behind it can retrieve them at answer time. A page it has never fetched cannot be quoted, summarized, or linked in that product's answers, so most sites keep it allowed to stay citable. Blocking it removes your pages from that AI surface and hands those citations to competitors.

How to handle TwinAgent

Allow TwinAgent when people in your organization or your customers intentionally use Twin to operate your site. If automated browser actions are not permitted, block the agent at the CDN, firewall, or application layer after checking its signature.

The stable token lets you publish a robots.txt rule:

User-agent: TwinAgent
Disallow: /

That rule expresses your preference, but compliance is not confirmed and Cloudflare's record says TwinAgent does not follow robots.txt. Use an enforced access rule when the block must hold.

For an allowlist, validate Web Bot Auth HTTP message signatures with the keys at https://signature-agents.twin.so/.well-known/http-message-signatures-directory. Do not grant access solely because a request contains TwinAgent in its user-agent header.

Examples

A finance team runs a Twin workflow that enters approved data into a browser-only vendor portal, and the portal verifies the request signature before allowing the agent through.
A SaaS company blocks TwinAgent at its edge because its terms do not permit automated interaction with account settings.
An analyst sees `TwinAgent/1.0` in access logs and classifies it as workflow automation rather than search-engine or training traffic.

Frequently asked questions about TwinAgent

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

It is associated with Twin's automation agents. When a workflow needs the Web Agent because an API cannot perform a required step, Twin can open the site in an isolated cloud browser, inspect the page, click, fill forms, or extract information.

Related bots

Amazon Kendra

Amazon Kendra is a managed information retrieval and intelligent search service that uses natural language processing and advanced deep learning model.

AI Assistant