Does Amzn-SearchBot respect robots.txt?

Amazon's current documentation says yes. Amzn-SearchBot honors specific Allow and Disallow rules and can inherit directives for other search bots when it has no named group.

Does Amzn-SearchBot collect generative AI training data?

No. Amazon explicitly says this crawler does not collect content for generative AI model training. Amazonbot has a different stated purpose and must be controlled separately.

Is Amzn-SearchBot the same as Amzn-User?

No. Amzn-SearchBot performs automated search crawling. Amzn-User may fetch live information for a user action and may not follow every robots.txt directive.

Does Amzn-SearchBot support crawl delay?

No. Amazon says its listed crawlers do not support the `Crawl-delay` directive. Use allow and disallow rules or server-side rate controls instead.

How can I verify Amzn-SearchBot traffic?

Look for `Amzn-SearchBot` in the user agent and compare the source address with Amazon's dedicated SearchBot IP list. Do not pin a rule to the changing Chrome version in the full user-agent string.

Amzn-SearchBot: AI Crawler Guide & robots.txt

What is Amzn-SearchBot?

Amzn-SearchBot is Amazon's crawler for improving search experiences in products and services such as Alexa and Rufus. Amazon says that allowing it makes a site's content eligible to appear in those search experiences. It is one member of Amazon's published bot family, with its own purpose and control setting.

The crawler is not Amazon's generative model training bot. Amazon explicitly says Amzn-SearchBot does not collect content for generative AI model training. Amazonbot may be used to improve products and train Amazon AI models, while Amzn-User fetches current information in response to a person's action. Rules for those user agents are independent.

Eligibility is the direct search implication. If Amzn-SearchBot can crawl a useful page, Amazon can consider that content for an applicable search result or Alexa response. A request does not guarantee that Amazon indexed the page, selected it for an answer, or will send a referral.

The documented user-agent format contains Amzn-SearchBot/0.1 inside a Chrome-style browser string. The stable matching token for robots.txt and logs is Amzn-SearchBot; matching the full browser version would be brittle because Chrome version numbers can change.

Amazon's current webmaster documentation says automated crawling by Amzn-SearchBot honors user-agent-specific Allow and Disallow directives. If a file does not name Amzn-SearchBot but permits other search bots, Amazon says this crawler follows those search-bot rules. It also honors page-level noindex, noarchive, and none, plus link-level nofollow, but it does not support Crawl-delay.

Amazon publishes a dedicated IP address list for Amzn-SearchBot at https://developer.amazon.com/amazonbot/searchbot-ip-addresses/. That gives operators another identity check beyond the user-agent string, although this bot is not marked as IP verified and no HTTP signature method is listed. Use Amazon's published list as the source when maintaining a firewall rule.

Relevant for AI searchAmazon

Is Amzn-SearchBot relevant for AI search?

Yes. Amzn-SearchBot feeds Amazon, so the pages it can reach shape what those AI products say about you.

Amzn-SearchBot crawls and indexes pages so the AI search or assistant behind it can retrieve them at answer time. A page it has never fetched cannot be quoted, summarized, or linked in that product's answers, so most sites keep it allowed to stay citable. Blocking it removes your pages from that AI surface and hands those citations to competitors.

How to handle Amzn-SearchBot

Keep Amzn-SearchBot allowed where eligibility for Amazon search products is wanted. Block it when that distribution is outside your content policy, or use narrower Disallow paths for sections that should not be indexed.

User-agent: Amzn-SearchBot
Disallow: /

Amazon says each of its bot settings is independent and that a change may take about 24 hours to reach its systems. Do not use Crawl-delay for this bot because Amazon does not support it. If network verification is needed, compare source addresses with Amazon's current SearchBot list instead of trusting only the Chrome-like user agent.

Examples

A recipe publisher allows Amzn-SearchBot so eligible instructions can be considered for Alexa search experiences, but excludes account pages.
A documentation site blocks Amzn-SearchBot from prerelease manuals while leaving released product guides crawlable.
A network administrator checks a claimed Amzn-SearchBot address against Amazon's published SearchBot range before creating a firewall exception.

Frequently asked questions about Amzn-SearchBot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

Amazon uses the crawler to improve search experiences across its products and services. Its documentation names Alexa as an example, and this entry also associates it with Rufus.

Related bots

Amazon Product Discovery

Amazon Product Discovery collects public product details from Amazon selling-partner, brand, and retailer websites.

Search Engine Crawler

Amazon Seller Initiated Listing

Amazon's web crawler that helps sellers succeed by giving them the option to provide a URL to a website and create high-quality product pages in Amazon's store.

E-commerce