Who operates Siteimprove Crawl?

Siteimprove Crawl is operated by Siteimprove. It functions as Siteimprove's SEO crawler.

Does Siteimprove Crawl respect robots.txt?

Siteimprove has not published a robots.txt commitment for Siteimprove Crawl. Treat its behavior as unconfirmed and watch your access logs to see whether your rules are honored.

Should I block Siteimprove Crawl?

It depends on whether you rely on the service behind Siteimprove Crawl. If your team or your agency uses it, keep it allowed; otherwise you can disallow it in robots.txt to save crawl budget.

Siteimprove Crawl - What It Is & How to Handle It

What is Siteimprove Crawl?

Siteimprove Crawl is Siteimprove's SEO crawler. Siteimprove content suite (i.e. Quality Assurance, Accessibility, Policy, and SEO). Crawls run on ports are 80 for HTTP and 443 for HTTPS.

Siteimprove Crawl is an operational bot rather than an AI crawler, so it does not directly affect whether you appear in AI answers. It still shows up in your logs, and bots like it help keep your site healthy and crawlable, which indirectly supports both SEO and AI visibility.

Like any automated client, Siteimprove Crawl consumes crawl budget and appears in your server and CDN logs. Reviewing those logs alongside your robots.txt rules helps you keep automated traffic deliberate and easy to reason about.

Want to see every AI bot hitting your site? Promptwatch turns your server and CDN logs into a live view of AI crawler and agent traffic, so you can watch ChatGPT, Claude, Perplexity, Gemini, and others crawl your pages and connect those visits to real citations and revenue. Learn more in AI crawler logs.

See every AI bot hitting your site

Promptwatch turns your server and CDN logs into a live view of AI crawler and agent traffic. Watch ChatGPT, Claude, Perplexity, Gemini, and more crawl your pages in real time, see exactly what they take, and connect every crawl to the citations and revenue it drives.

How to handle Siteimprove Crawl

Whether to allow Siteimprove Crawl depends on whether you rely on the service behind it. Use a robots.txt rule to allow or disallow it.

To control Siteimprove Crawl, add a rule for its user agent to your robots.txt:

User-agent: SiteCheck-sitecrawl
Disallow: /

Siteimprove does not publish a robots.txt commitment for Siteimprove Crawl, so confirm how it behaves by watching your access logs.

Examples

A site owner spots Siteimprove Crawl in access logs and confirms it against the operator's documented user agent.
An operations team allows Siteimprove Crawl so the SEO reporting it powers stays accurate.

Related bots

LinkCheck

The Siteimprove LinkCheck crawler analyzes and monitors websites for quality assurance, SEO, and accessibility purposes, and keeps website content in line with brand guidelines and organizational policies.

SEO