Who operates AhrefsSiteAudit?

Ahrefs operates it for the Site Audit tool. Ahrefs customers use the crawler to inspect websites for technical and on-page SEO issues.

Does AhrefsSiteAudit respect robots.txt?

Yes by default, according to Ahrefs. A verified site owner can deliberately authorize the crawler to ignore robots.txt on that site, so compliance is conditional when that option is enabled.

Why did AhrefsSiteAudit send many requests together?

An audit works through the pages in its configured scope and can create a noticeable batch of traffic. The project owner can reduce the crawl rate or reschedule the run.

Is AhrefsSiteAudit the same as AhrefsBot?

No. AhrefsSiteAudit checks a site for a user's audit project. AhrefsBot maintains Ahrefs' wider link database and supplies the Yep search index.

Does this crawler send pages to AI training?

No training purpose is documented. Its output is a Site Audit report for the Ahrefs user who configured the crawl.

What happens if I block AhrefsSiteAudit?

The audit may omit blocked pages and miss issues there. Blocking this token does not remove the site from AhrefsBot's database or from any unrelated AI service.

AhrefsSiteAudit - What It Is & How to Handle It

What is AhrefsSiteAudit?

AhrefsSiteAudit appears when an Ahrefs user runs Site Audit against a website. The crawler loads and renders pages to find technical and on-page SEO problems, including broken links, performance issues, and security misconfigurations. It is tied to an audit project rather than Ahrefs' continuous crawl of the wider web.

The crawler reports those issues to the Ahrefs user who configured the project. It does not build a public AI answer index, and Ahrefs does not identify it as collecting foundation-model training data. Fixes prompted by an audit may make a site easier for many crawlers to access, but an AhrefsSiteAudit visit itself has no documented AI-search effect.

Requests tend to arrive as the configured audit works through its URL set. A site owner may initiate the run manually or schedule it in Ahrefs, so traffic can be concentrated around audit windows. Cloudflare records Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/) as an example header.

Ahrefs says both of its bots obey robots.txt and crawl-delay directives by default. Site Audit has one important exception: a verified site owner can authorize AhrefsSiteAudit to ignore robots.txt for that site so the audit can inspect sections normally hidden from crawlers. Cloudflare marks compliance false, which does not capture this default-plus-override behavior.

How to handle AhrefsSiteAudit

If your team is running the audit, allow the intended public scope and tune crawl speed in Ahrefs or with a supported crawl delay. A site that does not want this crawler can publish:

User-agent: AhrefsSiteAudit
Disallow: /

The bot follows robots.txt by default, but a verified site owner can enable Ahrefs' documented override. If it reaches a disallowed path, first check the Site Audit project and who verified the domain. Authentication remains necessary for any page that should not be exposed to an authorized audit.

Examples

A retailer schedules a weekly Site Audit, then sees AhrefsSiteAudit fetch product and category pages during the configured run.
A verified owner authorizes an audit of a staging section that ordinary crawlers cannot reach through robots.txt.
An operations team traces a morning request spike to an Ahrefs audit and lowers its crawl speed instead of blocking the tool.