What does Search Submission do with the crawl?

It uses the discovered pages to create a sitemap and submit it to conventional search engines. An incomplete crawl can produce an incomplete sitemap.

How does the TrustedSite scanner identify itself?

TrustedSite documents the full user agent `Mozilla/5.0 (compatible; TrustedSite/1.0; +http://www.trustedsite.com/crawler)`.

Does TrustedSite obey robots.txt?

The operator does not state a firm policy in its crawler help, while the crawler record says no. Publish a specific rule if needed, then confirm the behavior in logs.

Can TrustedSite requests be authenticated?

TrustedSite offers optional Web Bot Auth for supported hosts and CDNs. That provides more assurance than the user-agent string alone.

Does TrustedSite crawl for AI search or training?

No general AI use is documented. Search Submission concerns standard search-engine sitemaps; the other scans support TrustedSite's own customer services.

TrustedSite & AI Search: What to Know

What is TrustedSite?

TrustedSite scans websites enrolled in its certification, diagnostics, and Search Submission services. The company was formerly known as McAfee SECURE. This is operational validation for customer accounts, not academic research.

What it fetches depends on the enabled product. Certification and diagnostic scans need enough of the site to complete their checks. Search Submission crawls pages to create a sitemap for submission to conventional search engines, so blocking it can leave that sitemap incomplete.

The full documented user agent is Mozilla/5.0 (compatible; TrustedSite/1.0; +http://www.trustedsite.com/crawler). TrustedSite asks customers to allowlist this scanner when bot protection causes skipped pages or failed scans. It also offers optional Web Bot Auth for hosts that can verify signed crawler requests.

The operator's help pages do not make a clear robots.txt commitment, and the accompanying crawler record says exclusions are not followed. TrustedSite has no documented general model training role. Search Submission may help ordinary search engines discover customer pages, but that is separate from generative AI search.

Indirectly relevant

Is TrustedSite relevant for AI search?

Indirectly. TrustedSite has no AI product of its own, but its output can end up in the systems that AI answers draw on.

TrustedSite gathers pages for research corpora and web-measurement studies. Several widely used AI training datasets began as academic crawls, so pages collected for a study can later teach commercial models what your brand is.

How to handle TrustedSite

Customers should permit the scanner on the public paths required by their TrustedSite products. If the host or CDN supports Web Bot Auth, use it to verify TrustedSite traffic. A plain allowlist based only on TrustedSite/1.0 can admit a client that copied the name.

The stable token allows a direct exclusion request:

User-agent: TrustedSite
Disallow: /

Treat this as a preference because operator documentation does not confirm compliance and the crawler record reports noncompliance. Check subsequent requests and use an edge rule when a firm block is required. Before blocking a customer scan, consider that certification may fail, diagnostics may miss pages, and Search Submission may not produce a complete sitemap. Private routes still belong behind authentication.

Examples

A retailer's certification scan skips several pages after its bot firewall rejects `TrustedSite/1.0`. The team configures Web Bot Auth and reruns the scan with verifiable requests.
A site with no TrustedSite account sees the claimed token crawl product pages. The administrator checks that no service was enabled, records the request sources, and blocks the unwanted traffic at the edge.

Frequently asked questions about TrustedSite

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

The usual reason is that the site is enrolled in TrustedSite certification, diagnostics, or Search Submission. Check the TrustedSite account when the visit is unexpected.