Visible Amsterdam: Join us Sep 3
Promptwatch Logo

Kernel Search

Kernel's search crawler that indexes web pages for AI-powered search and retrieval.
Kernel Technologies, Inc.KernelSearchBot
AI SearchAI Crawler

What is Kernel Search?

Kernel Search is Kernel Technologies, Inc.'s AI search crawler. Kernel's search crawler that indexes web pages for AI-powered search and retrieval.

Kernel Search decides which of your pages can appear in the AI search experience it powers. It crawls and indexes content so that experience can retrieve, quote, and link to it, so anything Kernel Search cannot reach is effectively invisible there. That makes it worth allowing unless you have a specific reason to opt out of the surface.

Tracking which AI crawlers and agents reach your site, and what they do once there, is the foundation of generative engine optimization. See our guides to AI crawlers and robots.txt to control automated access and protect your AI search visibility.

Want to see every AI bot hitting your site? Promptwatch turns your server and CDN logs into a live view of AI crawler and agent traffic, so you can watch ChatGPT, Claude, Perplexity, Gemini, and others crawl your pages and connect those visits to real citations and revenue. Learn more in AI crawler logs.

See every AI bot hitting your site

Promptwatch turns your server and CDN logs into a live view of AI crawler and agent traffic. Watch ChatGPT, Claude, Perplexity, Gemini, and more crawl your pages in real time, see exactly what they take, and connect every crawl to the citations and revenue it drives.

How to handle Kernel Search

Most sites keep Kernel Search allowed, because blocking it can remove your pages from the AI answers and search results it powers. Only disallow it if you specifically want to opt out of that surface.

To control Kernel Search, add a rule for its user agent to your robots.txt:

User-agent: KernelSearchBot
Disallow: /

Kernel Technologies, Inc. does not publish a robots.txt commitment for Kernel Search, so confirm how it behaves by watching your access logs.

Kernel Search is a Cloudflare signed agent: it signs requests with Web Bot Auth and publishes its keys at https://search.bot.kernel.sh/.well-known/http-message-signatures-directory, so you can tell genuine traffic from anything spoofing its user agent.

Examples

  • After publishing a new comparison page, a marketer checks whether Kernel Search has fetched it so the content can appear in AI answers.
  • An SEO lead keeps Kernel Search allowed in robots.txt to preserve visibility in AI-powered search.

Related bots

AI Search External

Cloudflare AI Search is a managed service that lets you connect your data and easily build AI-powered search.

AI Search

Alphalens Bot

Indexes companies and their offerings to power more effective business discovery.

AI Search

Amzn-SearchBot

Amzn-SearchBot crawls for improving Amazon search experiences (Alexa, Rufus).

AI Search

AI Search

Cloudflare AI Search is a managed service that lets you connect your data and easily build AI-powered search.

AI Search

Direqt Anomura

Anomura is Direqt’s search crawler, it discovers and indexes pages their customers websites.

AI Search

Frequently asked questions about Kernel Search

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

Kernel Search is operated by Kernel Technologies, Inc.. It functions as Kernel Technologies, Inc.'s AI search crawler.

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard