What is cercadorbot?
cercadorbot is Direct Support / Website Managed's search engine crawler. Search engine crawler. Indexes Catalan-language pages only; never used to train models. 5s between requests.
cercadorbot does what search crawlers have done for decades: fetch pages, follow links, and feed the index that ranks results.
Blocking cercadorbot removes your pages from that engine's results. For most sites the call is easy: leave it allowed unless you want out of that engine altogether.
In your logs, cercadorbot identifies itself with the user-agent token cercadorbot. Direct Support / Website Managed has not published a robots.txt commitment for it, so treat a disallow rule as a request rather than a guarantee.
To go deeper, our glossary covers AI crawlers, generative engine optimization, and how to use robots.txt to control automated access. To see which AI crawlers actually reach your pages, and which pages they skip, look at your AI crawler logs.
