Definition
An AI search API is a web search service designed to be called by language models and AI agents rather than by people. It takes a query, often written by the model itself, and returns ranked passages or page text sized for a context window, with URLs the model can cite. Exa, Perplexity's Search API, Brave Search API, Tavily and Parallel sell this as a standalone service, and OpenAI, Anthropic and Google build it into their model APIs as a web search tool.
The two kinds work differently:
- Standalone APIs return search results that a developer passes to any model, usually through function calling or an MCP server. Exa runs its own index, built by ExaSearchBot. Perplexity opened its Search API in September 2025, saying its index tracks more than 200 billion unique URLs and handles 200 million queries a day. Brave says every result comes from its own independent index, with no fallback to another engine, and its LLM Context endpoint returns extracted passages rather than links (Brave). Tavily packages search, page extraction, crawling and cited research as LLM-ready content (Tavily). Parallel, which also runs its own index, became available in AWS Marketplace on November 11, 2026, with search from $1 per 1,000 requests (Parallel).
- Built-in tools let the model decide when to search and return an answer with citations. OpenAI's
web_searchtool in the Responses API adds inline URL citations plus a fuller list of sources consulted, and developers can restrict it to up to 100 allowed or blocked domains (OpenAI). Anthropic charges $10 per 1,000 searches plus tokens for its web search tool, which takes the same kind of domain lists (Anthropic). Gemini's Grounding with Google Search draws on Google's index, and for Gemini 3 and newer models costs $14 per 1,000 queries after 5,000 free a month (Google).
Renting a classic engine's raw results has mostly stopped being an option. Microsoft retired the Bing Search APIs on August 11, 2025 and pointed customers to Grounding with Bing Search in Azure AI Foundry, which returns a grounded model answer with citations instead of raw results (Microsoft). Google's Custom Search JSON API is closed to new customers, and existing customers have until January 1, 2027 to move (Google).
The output is what sets these APIs apart. An agent wants the passages that answer its query, not ten blue links, so providers split pages into spans and rank them separately (sub-document retrieval). The provider also changes the answer. In the Artificial Analysis Search Index, which runs the same model and agent harness with only the search API swapped, the model alone scored 33. As of November 9, 2026, Keenable Search (pro) led at 83, Exa and Perplexity followed at 80, Brave scored 75 and OpenAI's web search 74 (Artificial Analysis). Perplexity's open-source search_evals framework tests APIs the same way (see retrieval evaluation).
For visibility, the consequence is that every app built on one of these APIs sees the web through that provider's AI search index. Many coding agents, research assistants, support bots and other products built with AI agent frameworks do not run their own crawler, so a page missing from Exa's or Perplexity's index is missing from every app that queries them, however well it ranks on Google. Which crawler feeds which API:
- ExaSearchBot builds Exa's index. It follows a robots.txt group addressed to it, or your rules for major search engines if there is none, signs every request with Web Bot Auth, and drops a page after seeing
noindexon its next fetch. Fetches made on a user's request go out as ExaUser (Exa). - PerplexityBot builds the index behind both Perplexity's answer engine and its Search API.
- Googlebot feeds Gemini's Google Search grounding, and Bingbot feeds Grounding with Bing Search (see Bing index).
Promptwatch's agent analytics has tracked ExaSearchBot and Exa's live citation fetches since November 6, 2026. Exa publishes no IP ranges, so neither gets a Verified badge, and the robots.txt generator for AI now writes an ExaSearchBot group whenever you add search engine rules (changelog).
In practice: allow the index crawlers for pages you want agents to find, and check your logs for the user-triggered fetchers that show an app reading a page to answer someone. Put facts in plain HTML text near the question they answer, since these APIs hand passages to the model. You cannot see the downstream apps, but you can run your priority questions through the APIs directly and see whether your pages come back.
Examples of AI Search API
- A developer-tools company sees in its crawler logs that most Exa citation fetches land on its API reference pages, so it moves those pages to the top of its update schedule.
- A travel brand finds its robots.txt has no group for ExaSearchBot, so the bot follows its Googlebot rules instead of the default group. It adds a dedicated group that opens its destination guides to Exa and keeps booking pages blocked.
- A support-bot vendor restricts OpenAI's web search tool to 40 allowed domains, so a retailer missing from that list never appears in the bot's answers, whatever its Google ranking.
- An SEO lead runs 30 priority category questions through the Perplexity and Exa search APIs each month and logs which of the company's pages come back, as a proxy for agent visibility.
