What financial data does it collect?

The public description says current financial data, but it does not name the fields, source sites, or FactSet product that receives them.

Does Factset_spyderbot follow robots.txt?

The crawler directory marks compliance as false, and no operator instructions in the available material say otherwise. Enforce a required block outside robots.txt.

Can a site verify the crawler from its header?

No. The token can be copied, and the record provides neither an official IP range nor an HTTP signature directory.

Does this crawl affect AI search or training?

No AI use is documented. The bot record describes financial-data collection, not generative search indexing or model training.

Factset_spyderbot & AI Search: What to Know

What is Factset_spyderbot?

FactSet runs Factset_spyderbot to collect current financial data from websites. The crawler record identifies it as a Python Selenium scraper, so its requests come from browser automation rather than a simple feed client. That explains why the header resembles a normal desktop browser.

Public information about this bot stops short of naming the fields it extracts or the FactSet dataset that receives them. It also does not list the sites covered by the crawl or a revisit schedule. A log entry therefore establishes the claimed collector and general purpose, not the exact downstream use.

The recorded user agent mimics Chrome 90 on Windows and ends with factset_spyderbot. Use the trailing token for classification. No official IP range or request-signing method is listed, and the crawler directory marks robots.txt compliance as false.

Its stated effect is to help FactSet keep financial data current. There is no documented connection to AI search answers or language-model training. A visit should be assessed as financial-data collection unless a separate agreement with FactSet says more.

Indirectly relevant

Is Factset_spyderbot relevant for AI search?

Indirectly. Factset_spyderbot has no AI product of its own, but its output can end up in the systems that AI answers draw on.

Factset_spyderbot collects content from many sites and redistributes it through its own product. Aggregated copies can end up in datasets that AI systems later learn from, and some aggregators are themselves sources that AI answer engines draw on.

How to handle Factset_spyderbot

Ask the team responsible for investor relations or data licensing whether FactSet is expected to retrieve the affected pages. Blocking can interrupt a collection arrangement, but the public bot record does not identify which data product would be affected.

The stable token can carry a robots preference:

User-agent: factset_spyderbot
Disallow: /

The directory records the bot as not following robots.txt. Apply an application or edge rule if the restriction must be enforced, and match factset_spyderbot rather than the Chrome wrapper. With no published IP list or signature, do not create an allowlist solely from the header.

Examples

An investor-relations site sees the bot render a public financial table and checks its FactSet licensing arrangement before allowing the route through a bot filter.
A company with no expected FactSet collection finds `factset_spyderbot` repeatedly loading a costly report page and blocks that route at the edge after a robots rule has no effect.

Frequently asked questions about Factset_spyderbot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

FactSet identifies it as a Python Selenium crawler. The recorded header is based on Chrome 90, with `factset_spyderbot` appended as the distinguishing token.