Promptwatch Logo

Statabot

Statabot searches for stata.toc files and indexes their contents.
StataCorp LLCStatabot
Aggregator

What is Statabot?

Statabot maintains the web portion of StataCorp's built-in search command. It looks for stata.toc files on approved sites and indexes the entries they describe, such as user-written Stata resources. Those results appear alongside material from Stata's local keyword database.

This index is curated rather than discovered through open web crawling. StataCorp says a site must be submitted and approved before Statabot will index it, and explicitly calls the service non-organic search. Finding a stray Statabot request is therefore a reason to check whether a current or former site maintainer registered the URL.

On an enrolled site, the crawler can return periodically. StataCorp describes the traffic as standard HTTP/1.1 GET requests and says the user-agent header contains Statabot. The cataloged full string is Statabot/1.0 https://www.stata.com/support/statabot.

The resulting records help Stata users find Stata-related web material. They are not a general search-engine index, an AI answer source, or a language-model training corpus. Removing a site changes its availability through Stata's search command, with no documented effect on broader AI visibility.

Indirectly relevant

Is Statabot relevant for AI search?

Indirectly. Statabot has no AI product of its own, but its output can end up in the systems that AI answers draw on.

Statabot collects content from many sites and redistributes it through its own product. Aggregated copies can end up in datasets that AI systems later learn from, and some aggregators are themselves sources that AI answer engines draw on.

How to handle Statabot

Keep the crawler reachable if the site is an approved Stata resource and its stata.toc entries should remain searchable. To add or remove a site, StataCorp asks the owner to email [email protected] with the full URL and the preferred HTTP or HTTPS transport.

You can also publish this robots.txt preference for the stable token:

User-agent: Statabot
Disallow: /

StataCorp's support page does not say that Statabot honors robots.txt. Use the documented removal request when you want the index changed, and use a server rule only when the HTTP requests themselves must stop.

Examples

  • A university that distributes Stata commands keeps its approved `stata.toc` file public and recognizes the periodic `Statabot` requests as expected indexing traffic.
  • After a research group retires its package site, the administrator sends StataCorp the complete HTTPS URL and asks for removal from Stata search.

Frequently asked questions about Statabot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

It searches approved sites for `stata.toc` files and indexes their contents for Stata's `search` command.

Related bots

AirsoftdbBot

Airsoftdb is a search engine for airsoft prod.

Aggregator

Artsdata Crawler

Web crawler that collects publicly available LOD for arts and culture in Canada.

Aggregator

BestChange Bot

The BestChange bot downloads exchange rate information from 600 websites every 5 seconds.

Aggregator

BoardGamePrices Bot

Price comparison site for board games. Need to crawl store pages for participating stores. All stores give permission to be crawled.

Aggregator

Capital One Bot

Capital One Bot crawls dealer websites for getting the usage information for Capital One lead navigator button.

Aggregator

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard