OAI-SearchBot and ChatGPT-User

OpenAI: “OAI-SearchBot is used to surface websites in search results in ChatGPT’s search features.” A site that keeps it out does not appear in ChatGPT search answers, and after a change to robots.txt it can take about 24 hours for OpenAI’s systems to follow.

The token is OAI-SearchBot. The user agent, as OpenAI gives it in its example (the version number may change):

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot

When it fetches robots.txt itself, OpenAI says it may add robots.txt; before the link, so those requests are easy to tell apart in logs that do not record the path.

When someone asks ChatGPT, or a custom GPT, something that needs a page, ChatGPT may open it as ChatGPT-User. OpenAI says it is not used to crawl the web automatically and not used to decide what appears in search. Because a person starts these fetches, OpenAI says robots.txt rules “may not apply”.

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot

With nothing about them in your file, both are allowed. To say it out loud, and keep training out at the same time:

User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: GPTBot
Disallow: /

OpenAI says each setting is independent of the others, so this is a combination it expects. Blocking OAI-SearchBot, if you want to, is the same two lines with Disallow: /.

• A wildcard that closes everything. User-agent: * with Disallow: /, then groups that let in Googlebot and Bingbot. OAI-SearchBot has no group of its own, so it reads the * group and stays out.
• One group for every AI bot. A list of AI user agents above a single Disallow: /, copied to stop training, sometimes has OAI-SearchBot and ChatGPT-User in it too.
• The edge, not the file. A firewall or Cloudflare’s AI bot settings can refuse them before robots.txt has a say. OpenAI publishes their addresses so a rule can let them through: searchbot.json and chatgpt-user.json. On Cloudflare: what the AI bot settings do.

The Trakis audit reports the first two as AI-005, with the line that does it.

Paste your file into the robots.txt validator, opened with OAI-SearchBot filled in. It runs in your browser and shows, for OAI-SearchBot and every other search and AI bot, whether it may come in and the exact line that decides. The list of AI crawlers has the others.

Read on 14 October 2026; operators change these pages, so we read them again every month.

• OpenAI, Overview of OpenAI Crawlers
• OpenAI, OAI-SearchBot IP addresses (JSON)
• OpenAI, ChatGPT-User IP addresses (JSON)