Promptwatch Logo

Grok Search

Grok Search fetches web pages in real time to power Grok's search and answer features inside X and the Grok apps.
UnverifiablexAIxAI-Grok
AI AssistantSearch Engine Crawler

What is Grok Search?

Grok Search is xAI's AI assistant crawler. Grok Search fetches web pages in real time to power Grok's search and answer features inside X and the Grok apps.

Grok Search helps decide whether you show up in AI answers. It fetches and indexes pages so its assistant can cite them in responses, which means the content it can reach is the content that can be quoted, linked, and recommended to users. For generative engine optimization, keeping Grok Search allowed is important: block it and you effectively opt out of being cited on that AI surface.

Tracking which AI crawlers and agents reach your site, and what they do once there, is the foundation of generative engine optimization. See our guides to AI crawlers and robots.txt to control automated access and protect your AI search visibility.

Promptwatch tracks Grok Search in real time from your server and CDN logs, so you can see exactly when it visits, which pages it requests, and how those crawls relate to citations. Learn more in AI crawler logs.

Track Grok Search with Promptwatch

Promptwatch classifies Grok Search (xAI-Grok) in real time from your server and CDN logs. See exactly when it visits, which pages it requests, the status codes it gets, and how those crawls map to AI citations.

How to handle Grok Search

Most sites keep Grok Search allowed, because blocking it can remove your pages from the AI answers and search results it powers. Only disallow it if you specifically want to opt out of that surface.

To control Grok Search, add a rule for its user agent to your robots.txt:

User-agent: xAI-Grok
Disallow: /

Because Grok Search performs user-initiated or real-time fetches, it may not always honor robots.txt, so confirm its behavior by watching your access logs.

Examples

  • After publishing a new comparison page, a marketer checks whether Grok Search has fetched it so the content can appear in AI answers.
  • An SEO lead keeps Grok Search allowed in robots.txt to preserve visibility in AI-powered search.

Related bots

GrokBot

GrokBot is xAI's crawler used to gather web content for training the Grok family of models. xAI publishes limited documentation for it.

AI Crawler

Grok DeepSearch

Grok DeepSearch performs multi-step research across the web to answer complex Grok queries with cited sources.

AI Assistant

OAI-SearchBot

Indexes websites for inclusion in ChatGPT's search results; does not crawl content for AI model training.

AI Assistant

PerplexityBot

Indexes websites for inclusion in Perplexity's search results; does not crawl content for AI model training.

AI Assistant

Claude-SearchBot

Claude-SearchBot navigates the web to improve search result quality for users. It analyzes online content specifically to enhance the relevance and accuracy of search responses.

AI Assistant

Frequently asked questions about Grok Search

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

Grok Search is operated by xAI. It functions as xAI's AI assistant crawler.

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard