What happens to the cookies it finds?

Termly captures their details, categorizes them, and compiles the results into the customer's cookie report.

Can TermlyBot miss a cookie?

Yes. A cookie may require a login, popup acceptance, button click, or other action that the scanner does not perform.

Does TermlyBot respect robots.txt?

Yes. Termly says the scanner obeys robots.txt and publishes `TermlyBot` as the token for an explicit allow rule.

Is TermlyBot an AI crawler?

No. It scans for cookies and tracking technologies, with no documented AI search or training use.

TermlyBot: What It Is & How to Handle It

What is TermlyBot?

TermlyBot appears when a Termly customer runs the website cookie scanner. The crawler starts with the supplied homepage, follows internal links, and acts as a browser so it can observe cookies set by the site and by outside services.

A single scan can touch many pages and load the scripts that set cookies. Termly records delayed cookies when they appear during the scan, then compiles the findings into a cookie report for the customer. The TermlyBot marker in the full browser user agent identifies this scanner.

The report covers what the scanner could reach and trigger. Cookies behind a login, an unaccepted popup, or another interaction may be absent if TermlyBot does not perform that action. Changes to the site's internal links can also change which pages a later scan discovers.

This is a cookie-compliance crawler. Termly does not document any use of the scanned pages for AI search, answer generation, or model training. Allowing the scan keeps Termly's report more complete, but it does not raise the site's visibility in AI results.

Not relevant for AI search

Is TermlyBot relevant for AI search?

No. TermlyBot is not part of AI search or training, so allowing or blocking it does not change your AI visibility.

TermlyBot runs synthetic monitoring or uptime checks on a schedule. It is not collecting content for any index or model, so it has no bearing on search rankings or AI answers. You usually see it because your own team or a service you use set up a check.

How to handle TermlyBot

Keep TermlyBot reachable when the site uses Termly's scan results to maintain its cookie inventory. If a general crawler rule is stopping the scan, Termly recommends this explicit exception:

User-agent: TermlyBot
Allow: /

Termly documents robots.txt compliance for the scanner. Use Disallow: / with the same stable token when you want to block it, and pause the scan in Termly if future reports are no longer needed. A firewall or hosting rule can still reject the browser after robots.txt allows it. Diagnose that layer separately and avoid granting broader access than the public pages being scanned.

Examples

A marketing team adds a tag and starts another scan. TermlyBot reaches a page that loads the tag, observes its cookie, and includes it in the new report.
A broad robots policy prevents the cookie scan, so the site adds an explicit `TermlyBot` allow rule while leaving the restriction in place for other crawlers.

Frequently asked questions about TermlyBot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

A Termly customer supplied the site for a cookie scan. The crawler follows internal links to find cookies and tracking technologies on reachable pages.