What is ExaSearchBot?
ExaSearchBot is the crawler behind Exa's index. The Exa crawler page says its purpose is search and retrieval: connecting people and applications to your content and sending them back to your site. Exa documents the user agent as Mozilla/5.0 (compatible; ExaSearchBot/1.0; +https://crawler.exa.ai/), with ExaSearchBot as the robots.txt product token.
Every request is signed with HTTP Message Signatures (RFC 9421) following the Web Bot Auth scheme, with Signature, Signature-Input and Signature-Agent headers. Cloudflare, Akamai and other providers that support Web Bot Auth validate it automatically.
ExaSearchBot respects the Robots Exclusion Protocol. It looks for a group addressed to ExaSearchBot first, then follows the rules set for major search engines, and otherwise the * group. Exa also says it limits request rates per site and does not try to bypass logins, paywalls or CAPTCHAs.
Pages fetched live for a user or an app go out as ExaUser instead. robots.txt controls crawling, not indexing: to remove a page already in the index, serve a noindex meta tag or X-Robots-Tag: noindex header and let ExaSearchBot re-fetch it.
Promptwatch classifies ExaSearchBot in your AI crawler logs as an AI crawl: indexing traffic, kept apart from the citation fetches ExaUser makes. You can see which pages it reaches, which ones return errors, and which ones it has never requested. Exa publishes no IP ranges, so the match is on the user agent; the Web Bot Auth signature is the stronger check.
