What is CriteoBot?
CriteoBot appears when a site asks Criteo to deliver an ad and Criteo needs a current category for the page. The crawler reads public content, then labels the page for contextual advertising. Criteo's own example places an article about running shoes under sport and running.
The crawler does not need to fetch a URL for every ad request. Criteo says it returns when an existing classification is unavailable or out of date. It reads the published page, while data about the person viewing that page is outside this crawler's stated scope.
Requests use CriteoBot/0.1 and include a link to Criteo's crawler information. The operator publishes current network ranges and gives site owners robots.txt rules for allowing, limiting, or blocking access. Crawl-delay is supported up to 30 seconds, including decimal values.
The categories produced by CriteoBot are for ad placement. Criteo does not document the crawler as a source for AI answers or model training. A block can deprive Criteo advertisers of page context, but allowing the bot has no stated benefit for AI search visibility.
