What is PerplexityBot?
PerplexityBot is the automatic crawler Perplexity uses to surface and link websites in its search results. It gathers and indexes public web information ahead of user questions. Perplexity-User handles the separate case where a live question causes a page visit.
Site owners can direct PerplexityBot through robots.txt. Perplexity says the settings work independently and may take up to 24 hours to reflect a change. That makes the PerplexityBot group the appropriate place to choose which public paths are available to Perplexity's search crawler.
A crawl can occur without a person requesting that exact page at the same time. Responses, redirect targets, and accessible page content determine what the bot can retrieve. Seeing PerplexityBot in a server log confirms a fetch, but it does not show that Perplexity indexed, ranked, or cited the page.
Blocking this bot can keep the affected content from being surfaced through Perplexity search. Allowing it leaves that route open, which is useful for sites that want their articles or documentation available as answer sources. Eligibility is not a promise that Perplexity will use a page for a particular query.
Perplexity explicitly says PerplexityBot is not used to crawl content for AI foundation models. A robots decision for this token controls search crawling, not model training. The absence of a training purpose is part of Perplexity's published description of the agent.
Perplexity documents the full string as Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot). Its current network ranges are published at https://www.perplexity.com/perplexitybot.json. The operator's crawler documentation recommends combining the user agent and those ranges when configuring a WAF.
