Promptwatch Logo

AI Search API

An AI search API is a web search service built for LLMs and agents, returning ranked passages and URLs a model can read and cite, such as Exa or Brave.
Updated November 15, 2026
AI

Definition

An AI search API is a web search service designed to be called by language models and AI agents rather than by people. It takes a query, often written by the model itself, and returns ranked passages or page text sized for a context window, with URLs the model can cite. Exa, Perplexity's Search API, Brave Search API, Tavily and Parallel sell this as a standalone service, and OpenAI, Anthropic and Google build it into their model APIs as a web search tool.

The two kinds work differently:

  • Standalone APIs return search results that a developer passes to any model, usually through function calling or an MCP server. Exa runs its own index, built by ExaSearchBot. Perplexity opened its Search API in September 2025, saying its index tracks more than 200 billion unique URLs and handles 200 million queries a day. Brave says every result comes from its own independent index, with no fallback to another engine, and its LLM Context endpoint returns extracted passages rather than links (Brave). Tavily packages search, page extraction, crawling and cited research as LLM-ready content (Tavily). Parallel, which also runs its own index, became available in AWS Marketplace on November 11, 2026, with search from $1 per 1,000 requests (Parallel).
  • Built-in tools let the model decide when to search and return an answer with citations. OpenAI's web_search tool in the Responses API adds inline URL citations plus a fuller list of sources consulted, and developers can restrict it to up to 100 allowed or blocked domains (OpenAI). Anthropic charges $10 per 1,000 searches plus tokens for its web search tool, which takes the same kind of domain lists (Anthropic). Gemini's Grounding with Google Search draws on Google's index, and for Gemini 3 and newer models costs $14 per 1,000 queries after 5,000 free a month (Google).

Renting a classic engine's raw results has mostly stopped being an option. Microsoft retired the Bing Search APIs on August 11, 2025 and pointed customers to Grounding with Bing Search in Azure AI Foundry, which returns a grounded model answer with citations instead of raw results (Microsoft). Google's Custom Search JSON API is closed to new customers, and existing customers have until January 1, 2027 to move (Google).

The output is what sets these APIs apart. An agent wants the passages that answer its query, not ten blue links, so providers split pages into spans and rank them separately (sub-document retrieval). The provider also changes the answer. In the Artificial Analysis Search Index, which runs the same model and agent harness with only the search API swapped, the model alone scored 33. As of November 9, 2026, Keenable Search (pro) led at 83, Exa and Perplexity followed at 80, Brave scored 75 and OpenAI's web search 74 (Artificial Analysis). Perplexity's open-source search_evals framework tests APIs the same way (see retrieval evaluation).

For visibility, the consequence is that every app built on one of these APIs sees the web through that provider's AI search index. Many coding agents, research assistants, support bots and other products built with AI agent frameworks do not run their own crawler, so a page missing from Exa's or Perplexity's index is missing from every app that queries them, however well it ranks on Google. Which crawler feeds which API:

  • ExaSearchBot builds Exa's index. It follows a robots.txt group addressed to it, or your rules for major search engines if there is none, signs every request with Web Bot Auth, and drops a page after seeing noindex on its next fetch. Fetches made on a user's request go out as ExaUser (Exa).
  • PerplexityBot builds the index behind both Perplexity's answer engine and its Search API.
  • Googlebot feeds Gemini's Google Search grounding, and Bingbot feeds Grounding with Bing Search (see Bing index).

Promptwatch's agent analytics has tracked ExaSearchBot and Exa's live citation fetches since November 6, 2026. Exa publishes no IP ranges, so neither gets a Verified badge, and the robots.txt generator for AI now writes an ExaSearchBot group whenever you add search engine rules (changelog).

In practice: allow the index crawlers for pages you want agents to find, and check your logs for the user-triggered fetchers that show an app reading a page to answer someone. Put facts in plain HTML text near the question they answer, since these APIs hand passages to the model. You cannot see the downstream apps, but you can run your priority questions through the APIs directly and see whether your pages come back.

Examples of AI Search API

  • A developer-tools company sees in its crawler logs that most Exa citation fetches land on its API reference pages, so it moves those pages to the top of its update schedule.
  • A travel brand finds its robots.txt has no group for ExaSearchBot, so the bot follows its Googlebot rules instead of the default group. It adds a dedicated group that opens its destination guides to Exa and keeps booking pages blocked.
  • A support-bot vendor restricts OpenAI's web search tool to 40 allowed domains, so a retailer missing from that list never appears in the bot's answers, whatever its Google ranking.
  • An SEO lead runs 30 priority category questions through the Perplexity and Exa search APIs each month and logs which of the company's pages come back, as a proxy for agent visibility.

Terms related to AI Search API

AI Search Index

An AI search index is the store of crawled web pages an AI assistant retrieves from when it answers. Google, OpenAI, Perplexity and Meta each run their own.

AI

AI API

An AI API is a hosted interface for sending prompts, files and tool definitions to a provider's model, such as GPT-6, Claude or Gemini, and getting output back.

AI

AI Grounding

AI grounding ties a model's answer to verifiable sources, such as live search results or supplied documents, so each claim can be traced and cited.

AI

RAG (Retrieval-Augmented Generation)

RAG is how AI answer engines like ChatGPT search, Perplexity and AI Mode fetch web pages at question time and cite them, deciding which brands appear.

AI

Real-Time Search

Real-time search is retrieving current web information at the moment a question is asked, so an AI answer reflects today's facts rather than training data.

SEO

AI Agent Frameworks

AI agent frameworks are SDKs such as LangGraph, CrewAI, the OpenAI Agents SDK and Google ADK that handle tool calls, state and orchestration for AI agents.

AI

User-Triggered Fetchers

User-triggered fetchers are bots like ChatGPT-User and Perplexity-User that fetch a page when a person asks, and may ignore robots.txt rules in AI search.

AI

Function Calling / Tool Use

Function calling, or tool use, lets an LLM request a defined function, such as a search or API call, with structured arguments that an app then runs.

AI

Model Context Protocol (MCP)

Model Context Protocol (MCP) is an open standard for connecting AI apps to tools and data. Anthropic created it; the Linux Foundation's AAIF now hosts it.

AI

Sub-Document Retrieval

Sub-document retrieval is when AI search systems index and rank passages within pages rather than whole pages, making the passage the unit that gets cited.

GEO

Retrieval Evaluation

Retrieval evaluation measures whether a search or AI system finds the right documents and passages for a query, before judging the answer written from them.

Analytics

Web Bot Auth

Web Bot Auth is an IETF draft protocol in which bots sign each HTTP request with a private key, so websites can verify which operator actually sent it.

AI

Frequently Asked Questions about AI Search API

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard