Trakis

AI crawlers

PerplexityBot and Perplexity-User

Perplexity has two agents, and both are about answers, not training: PerplexityBot finds and links pages in Perplexity’s results, Perplexity-User opens a page someone asked about. Blocking PerplexityBot keeps your pages out of Perplexity’s results; it keeps them out of no training set, because neither agent is used for one.

PerplexityBot: Perplexity's results

Perplexity says PerplexityBot is designed to surface and link websites in its search results, and is not used to crawl content for AI foundation models. It recommends allowing it in robots.txt for a site to appear there.

The token is PerplexityBot. The user agent:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)

Perplexity-User: a page someone asked about

Perplexity-User visits a page when a person’s question needs it, for an answer that may cite it. It is not used to crawl the web or to collect training data. Because a person asked, Perplexity says this fetcher “generally ignores robots.txt rules”.

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Perplexity-User/1.0; +https://perplexity.ai/perplexity-user)

Let them in, or keep them out

With nothing about them in your file, both are allowed. To keep PerplexityBot out:

User-agent: PerplexityBot
Disallow: /

Perplexity says each setting works on its own, and that changes can take up to 24 hours to show in its systems. For Perplexity-User, robots.txt is a request it says it generally does not follow; a firewall rule is what decides.

Behind a firewall

  • Perplexity asks sites behind a firewall to allow both agents by name and by address, from its two lists: perplexitybot.json and perplexity-user.json.
  • A user agent alone proves nothing; anyone can send it. The address is what tells the real bot from a copy.
  • On Cloudflare, the AI bot settings can refuse them at the edge whatever robots.txt says: what those settings do.

The Trakis audit reports robots.txt keeping PerplexityBot out of the site as AI-005.

Check what your robots.txt tells PerplexityBot

Paste your file into the robots.txt validator, opened with PerplexityBot filled in. It runs in your browser and shows, for PerplexityBot and every other search and AI bot, whether it may come in and the exact line that decides. The list of AI crawlers has the others.

Sources

Read on 14 October 2026; operators change these pages, so we read them again every month.

Last changed 14 October 2026.