Live session Nov 5: Fireflies' AI Search Playbook
Promptwatch Logo

OctenSearchBot

Octen's search crawler, which indexes the web for the search service AI applications use.
OctenOctenSearchBot
AI SearchAI Crawler

What is OctenSearchBot?

OctenSearchBot is the crawler for Octen. The Octen crawler page documents the user agent as Mozilla/5.0 (compatible; OctenSearchBot/1.0; +https://crawler.octen.ai/) and says its robots.txt product token is OctenSearchBot.

Every request is signed with HTTP Message Signatures (RFC 9421) following Web Bot Auth, with Signature, Signature-Input and Signature-Agent headers. Octen publishes its public keys at https://crawler.octen.ai/.well-known/http-message-signatures-directory, and says verification holds regardless of the request's IP address.

OctenSearchBot respects the Robots Exclusion Protocol.

Relevant for AI search

Is OctenSearchBot relevant for AI search?

Yes. OctenSearchBot builds the web index behind Octen, a search service AI applications use.

Octen provides web search to AI applications, and OctenSearchBot is the crawler that feeds it. A page it has not fetched cannot be returned in those searches, so blocking it removes you from the agents and apps that rely on Octen.

How to handle OctenSearchBot

Keep OctenSearchBot allowed if you want AI applications that search through Octen to find your pages. To block it:

User-agent: OctenSearchBot
Disallow: /

Use the Web Bot Auth signature, not the user-agent string, to confirm a request is genuine.

Examples

  • A site owner checks that signed OctenSearchBot requests reach a newly published guide.
  • A publisher adds an `OctenSearchBot` group to keep a members-only section out of the index.

Frequently asked questions about OctenSearchBot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

Yes. Octen says it respects the Robots Exclusion Protocol and names `OctenSearchBot` as its product token.

Related bots

ExaSearchBot

Exa's search crawler, which discovers and indexes public web pages for its search and retrieval API.

AI Search

KeenableBot

Keenable's crawler for public pages in its search index, which answers queries from AI agents with linked sources.

AI Search

ShapBot

Crawls and indexes web content to power Parallel's search and content extraction APIs for AI applications.

AI Crawler

AIWebIndex

Lyrenth's scheduled crawler that indexes the public web and serves pages to AI agents as attributed documents.

AI Search

LinkupBot

LinkupBot discovers and retrieves publicly accessible webpages to build and maintain the Linkup web search index.

Search Engine Crawler

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard