Promptwatch Logo

Amazonbot

Amazonbot is Amazon's general web crawler for improving Amazon products and services.
Amazonbot
AI CrawlerAI Training

What is Amazonbot?

Amazonbot is Amazon's general web crawler for improving Amazon products and services. The directory description specifically mentions more accurate Alexa answers, while Amazon's current crawler documentation says Amazonbot data may also be used to train Amazon AI models.

Amazon now documents three separate web identities. Amazonbot is the general improvement and possible training crawler. Amzn-SearchBot supports search experiences such as Alexa, and Amzn-User performs live fetches for a person's request. A rule for Amazonbot does not automatically control those other two user agents.

Amazonbot identifies itself with the stable Amazonbot token and a version such as Amazonbot/0.1. Amazon says its automated crawling checks a host-level robots.txt file, honors allow and disallow directives, and may use a cached copy from the previous 30 days. It does not support crawl-delay.

Page-level controls offer a more specific choice. Amazon says noarchive prevents a page from being used for model training, noindex prevents indexing, and none prevents indexing. The crawler also respects rel=nofollow on links. These controls can be useful when a site accepts crawling but wants to limit later use.

Allowing Amazonbot makes public content available to Amazon's documented product-improvement and possible training workflow. It does not guarantee an Alexa citation or recommendation. Blocking Amazonbot stops compliant future crawling by that identity, but it is not a complete opt-out from Amazon search because Amzn-SearchBot is independent.

Cloudflare's verified-bot record identifies Amazon as the operator, links to the same developer page, and marks Amazonbot as following robots.txt. Amazon also publishes crawler IP addresses at https://developer.amazon.com/amazonbot/ip-addresses/. this bot is not marked IP verified and has no Web Bot Auth key directory, so compare claimed traffic with Amazon's current list rather than trusting the header alone.

Relevant for AI searchAmazon QAlexa

Is Amazonbot relevant for AI search?

Yes. Amazonbot feeds Amazon Q, Alexa, so the pages it can reach shape what those AI products say about you.

Amazonbot gathers public web content that can end up in the training data for large language models. Once your pages are in that set, they influence how the operator's models talk about you for that model generation. Allowing it lets your own writing carry weight in those answers; blocking it means the models learn about you from third parties instead.

How to handle Amazonbot

Disallow Amazonbot if you do not want this general Amazon crawler accessing the site. Keep it allowed if Amazon's documented product-improvement and possible model-training uses are acceptable.

The site-wide robots.txt rule is:

User-agent: Amazonbot
Disallow: /

Amazon says Amazonbot follows allow and disallow rules, though robots.txt may be served from a cached copy. Do not expect a newly published rule to change every request immediately, and remember that Amzn-SearchBot and Amzn-User need their own groups.

For page-level training control, Amazon documents noarchive as an instruction not to use the page for model training. Review the effect of a generic robots meta tag on other crawlers before deploying it. Validate suspicious traffic against Amazon's published IP address file, since Amazonbot is easy to copy.

Examples

  • A publisher permits Amazonbot to crawl public reference pages but marks selected articles `noarchive` because it does not want those pages used for Amazon model training.
  • An infrastructure team compares a claimed Amazonbot source with Amazon's published address file before changing a rate-limit rule.
  • A webmaster blocks Amazonbot and separately reviews Amzn-SearchBot, knowing the first rule does not cover Alexa search crawling.

Frequently asked questions about Amazonbot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

Amazon operates it. The metadata's operator field is blank, but Amazon's developer documentation and Cloudflare's verified-bot record both identify Amazon as the operator.

Related bots

AI2Bot

AI2Bot is operated by the Allen Institute for Artificial Intelligence (Ai2) to crawl the web for content to train open-source AI models.

AI Crawler

aiHitBot

aiHitBot collects and maintains historical information about companies.

AI Crawler

Applebot

Applebot powers search features in Apple's ecosystem (Spotlight, Siri, Safari) and may be used to train Apple's foundation models for generative AI features.

AI Crawler

Atlassian Rovo

Crawls and indexes web content for Atlassian Rovo's AI-powered search, chat, and agents.

AI Crawler

Browserbase

Runs headless browser automation on behalf of Browserbase customers for web scraping, form submission, and testing.

AI Crawler

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard