Does Spyglasses respect robots.txt?

Its operator has not published a robots.txt commitment for Spyglasses. Treat its behavior as unconfirmed and watch your access logs to see whether your rules are honored.

Should I block Spyglasses?

It depends on whether you rely on the service behind Spyglasses. If your team or your agency uses it, keep it allowed; otherwise you can disallow it in robots.txt to save crawl budget.

Visible Amsterdam: Join us Sep 3Visible Amsterdam: Join us on September 3 for an exclusive afternoon exploring the future of AI Search. Request your spot now

Book a Demo

Spyglasses

Q: How can I verify Spyglasses is genuine?

Spyglasses is a Cloudflare signed agent. It signs requests using Web Bot Auth HTTP message signatures and publishes its public keys at https://www.spyglasses.io/.well-known/http-message-signatures-directory, so you can validate a request at your edge instead of trusting the user-agent string, which anyone can spoof.

Spyglasses accesses site content to assist with AEO capabilities. Learn more at https://www.spyglasses.io.

SpyglassesSpyglassesBot

SEO

What is Spyglasses?

Spyglasses is a SEO crawler. Spyglasses accesses site content to assist with AEO capabilities. Learn more at https://www.spyglasses.io.

Spyglasses is an operational bot rather than an AI crawler, so it does not directly affect whether you appear in AI answers. It still shows up in your logs, and bots like it help keep your site healthy and crawlable, which indirectly supports both SEO and AI visibility.

Like any automated client, Spyglasses consumes crawl budget and appears in your server and CDN logs. Reviewing those logs alongside your robots.txt rules helps you keep automated traffic deliberate and easy to reason about.

Want to see every AI bot hitting your site? Promptwatch turns your server and CDN logs into a live view of AI crawler and agent traffic, so you can watch ChatGPT, Claude, Perplexity, Gemini, and others crawl your pages and connect those visits to real citations and revenue. Learn more in AI crawler logs.

See every AI bot hitting your site

Promptwatch turns your server and CDN logs into a live view of AI crawler and agent traffic. Watch ChatGPT, Claude, Perplexity, Gemini, and more crawl your pages in real time, see exactly what they take, and connect every crawl to the citations and revenue it drives.

How to handle Spyglasses

Whether to allow Spyglasses depends on whether you rely on the service behind it. Use a robots.txt rule to allow or disallow it.

To control Spyglasses, add a rule for its user agent to your robots.txt:

User-agent: SpyglassesBot
Disallow: /

The operator does not publish a robots.txt commitment for Spyglasses, so confirm how it behaves by watching your access logs.

Spyglasses is a Cloudflare signed agent: it signs requests with Web Bot Auth and publishes its keys at https://www.spyglasses.io/.well-known/http-message-signatures-directory, so you can tell genuine traffic from anything spoofing its user agent.

Examples

A site owner spots Spyglasses in access logs and confirms it against the operator's documented user agent.
An operations team allows Spyglasses so the SEO reporting it powers stays accurate.

Related bots

AhrefsBot

Powers the database for both Ahrefs, a marketing intelligence platform, and Yep, an independent, privacy-focused search engine.

SEO

AhrefsSiteAudit

Powers Ahrefs’ Site Audit tool. Ahrefs users can use Site Audit to analyze websites and find both technical SEO and on-page SEO issues.

SEO

Barkrowler

Barkrowler is Babbar's web crawler that fuels and updates their graph representation of the web, providing SEO tools for the marketing community.

SEO

ClarityBot

ClarityBot is seoClarity's web crawler that performs technical SEO audits, analyzes content, and monitors website performance.

SEO

DataForSeoBot

DataForSeoBot is a backlink checker bot operated by DataForSEO that crawls websites to build and maintain their backlink database.

SEO

Frequently asked questions about Spyglasses

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

Spyglasses is operated by the company of the same name. It functions as a SEO crawler.

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.