Who operates Semrush Site Audit?

Semrush Site Audit is an SEO crawler; its operator is identified in its user-agent string and public documentation.

Does Semrush Site Audit respect robots.txt?

Yes. Semrush Site Audit generally follows robots.txt directives, so you can allow or disallow it with a rule for its user agent.

Should I block Semrush Site Audit?

It depends on whether you rely on the service behind Semrush Site Audit. If you do, keep it allowed; otherwise you can disallow it in robots.txt.

Semrush Site Audit - What It Is & How to Handle It

What is Semrush Site Audit?

Semrush Site Audit is an SEO crawler. Semrush Site Audit is a powerful website crawler that analyzes the health of a website by checking for on-page and technical SEO issues, including duplicate content, broken links, HTTPS implementation, hreflang attributes, and more.

Semrush Site Audit is an operational bot rather than an AI crawler, so it does not directly affect whether you appear in AI answers. It still shows up in your logs, and bots like it help keep your site healthy and crawlable, which indirectly supports both SEO and AI visibility.

Like any automated client, Semrush Site Audit consumes crawl budget and appears in your server and CDN logs. Reviewing those logs alongside your robots.txt rules helps you keep automated traffic deliberate and easy to reason about.

Want to see every AI bot hitting your site? Promptwatch turns your server and CDN logs into a live view of AI crawler and agent traffic, so you can watch ChatGPT, Claude, Perplexity, Gemini, and others crawl your pages and connect those visits to real citations and revenue. Learn more in AI crawler logs.

See every AI bot hitting your site

Promptwatch turns your server and CDN logs into a live view of AI crawler and agent traffic. Watch ChatGPT, Claude, Perplexity, Gemini, and more crawl your pages in real time, see exactly what they take, and connect every crawl to the citations and revenue it drives.

How to handle Semrush Site Audit

Whether to allow Semrush Site Audit depends on whether you rely on the service behind it. Use a robots.txt rule to allow or disallow it.

To control Semrush Site Audit, add a rule for its user agent to your robots.txt:

User-agent: SemrushSiteAudit
Disallow: /

Replace the user-agent token above with the exact string Semrush Site Audit sends, check the operator's documentation, since display names and user-agent tokens can differ.

Semrush Site Audit generally honors robots.txt directives.

Examples

A site owner spots Semrush Site Audit in access logs and confirms it against the operator's documented user agent.
An operations team allows Semrush Site Audit so the service it powers keeps working as expected.