What is the difference between GPTBot and ChatGPT-User?

GPTBot is OpenAI's bulk crawler that collects content used to train future models; it crawls broadly and on its own schedule, unprompted by any user. ChatGPT-User is a live fetcher that requests a page only because a person in a ChatGPT conversation asked something that requires it. Blocking GPTBot withholds your content from training. Blocking ChatGPT-User prevents ChatGPT from reading your page during a real user's question, which is the traffic most likely to produce a citation.

Should I block GPTBot?

It depends on what you sell. If your content is the product — a paywalled archive, a subscription research library — blocking training crawlers protects the asset. If your content is marketing for something else, blocking GPTBot removes you from the corpus that shapes what models say about your category, with no offsetting benefit. Most SaaS companies should allow GPTBot on marketing and docs, and block it on application routes.

Does blocking GPTBot stop ChatGPT from citing my site?

Not directly, because ChatGPT-User and OAI-SearchBot are separate agents governed by separate robots.txt directives. But it does remove your content from the training corpus, so the model is less likely to know your product exists well enough to go looking for it.

Blog Crawlers

GPTBot vs ChatGPT-User vs OAI-SearchBot: which one is reading your site

OpenAI operates three distinct crawlers with three different purposes. Blocking the wrong one costs you citations; allowing the wrong one costs you leverage. Here is how to tell them apart in your logs.

June 12, 2026 · Updated August 7, 2026 Markdown for agents

OpenAI does not have “a crawler”. It has three, and they answer to different directives, arrive with different frequencies, and mean completely different things when they show up in your logs. Teams that treat them as one entity make one of two mistakes: they block everything and quietly disappear from AI answers, or they allow everything and hand over content they meant to keep.

The three agents

Agent	User-agent token	Triggered by	What it means when you see it
Training crawler	`GPTBot`	OpenAI’s own schedule	Your page may enter the corpus for future models
Live fetcher	`ChatGPT-User`	A person’s question, right now	ChatGPT is reading your page to answer someone
Search indexer	`OAI-SearchBot`	Index maintenance	Your page is being indexed for ChatGPT search results

They are separately addressable in robots.txt, which is the whole point: OpenAI’s design lets you make different decisions about training and about retrieval.

GPTBot

GPTBot is the bulk crawler. It fetches broadly, at its own cadence, and its output feeds model training. No human triggered any individual request. In logs it looks like classic crawling: many URLs, systematic path coverage, steady rate, and no correlation with your human traffic pattern.

Full user-agent string, roughly:

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot

ChatGPT-User

ChatGPT-User is the interesting one. It fetches a page because someone in a ChatGPT conversation asked a question the model decided your page could answer — a browsing request, a link the user pasted, or a tool call during a search. Each hit is downstream of an actual human intent.

That makes ChatGPT-User traffic the highest-signal machine traffic you can measure. It is spiky rather than systematic, concentrated on a handful of URLs, and often correlated with topical interest. If your pricing page is getting repeated ChatGPT-User hits, people are asking ChatGPT what you cost.

OAI-SearchBot

OAI-SearchBot builds and maintains the index behind ChatGPT’s search results. It is closer in spirit to Googlebot than to either of the others: allowing it is how you become findable inside ChatGPT search, and it does not feed training.

The same split exists across operators

The pattern is not OpenAI-specific. Most operators separate bulk collection from live retrieval:

Anthropic — ClaudeBot (crawling for training), Claude-User (fetching during a conversation), Claude-SearchBot (search index).
Perplexity — PerplexityBot (index), Perplexity-User (live fetch during an answer).
Google — Googlebot (search index), Google-Extended (a robots token controlling Gemini training use; it is not a separate crawler).
Apple — Applebot (Siri and Spotlight), Applebot-Extended (training opt-out token).
Common Crawl — CCBot, a nonprofit archive that many downstream training datasets are built from.

Google-Extended and Applebot-Extended are worth calling out because they behave differently from the rest: they are not user agents that will ever appear in your logs. They are tokens you place in robots.txt to change how content already crawled by Googlebot or Applebot may be used. You cannot measure them; you can only set them.

How to decide

Treat it as two independent questions, not one.

Question one: do you want to be in the training corpus? Being in training is why a model knows your category, your product name, and roughly what you do, without having to look anything up. For most B2B companies that is straightforwardly good — it is the AI-era equivalent of brand awareness. It is bad when your content is the product: a research archive, a paid dataset, an editorial library that people subscribe to.

Question two: do you want to be retrievable during a live answer? Almost always yes. This is where citations come from, and citations are where referral traffic comes from. Blocking ChatGPT-User, Claude-User, or Perplexity-User is a decision to be absent from answers about your own category — usually made accidentally, by a wildcard rule.

A common resolution for a SaaS company:

User-agent: GPTBot
Allow: /blog/
Allow: /docs/
Disallow: /app/
Disallow: /api/

User-agent: ChatGPT-User
Allow: /

User-agent: OAI-SearchBot
Allow: /

Reading it in your logs

Two practical warnings.

User agents are trivially spoofable. Anyone can send User-Agent: GPTBot. OpenAI, Anthropic, and Perplexity all publish IP ranges for verification, and serious traffic decisions should verify against them rather than trusting the string. Scrapers impersonating well-known agents to bypass bot rules is common enough that unverified counts will overstate your reach.

Your analytics cannot see any of this. None of these agents execute JavaScript, so none of them appear in Google Analytics or any client-side tool. The data exists only in server logs or at whatever sits in front of your origin. If you have never looked, the honest position is that you do not currently know which of these agents read your site. Measuring AI crawler traffic covers how to fix that, and robots.txt for AI crawlers covers the directive syntax in full.

Common questions

What is the difference between GPTBot and ChatGPT-User?: GPTBot is OpenAI's bulk crawler that collects content used to train future models; it crawls broadly and on its own schedule, unprompted by any user. ChatGPT-User is a live fetcher that requests a page only because a person in a ChatGPT conversation asked something that requires it. Blocking GPTBot withholds your content from training. Blocking ChatGPT-User prevents ChatGPT from reading your page during a real user's question, which is the traffic most likely to produce a citation.
Should I block GPTBot?: It depends on what you sell. If your content is the product — a paywalled archive, a subscription research library — blocking training crawlers protects the asset. If your content is marketing for something else, blocking GPTBot removes you from the corpus that shapes what models say about your category, with no offsetting benefit. Most SaaS companies should allow GPTBot on marketing and docs, and block it on application routes.
Does blocking GPTBot stop ChatGPT from citing my site?: Not directly, because ChatGPT-User and OAI-SearchBot are separate agents governed by separate robots.txt directives. But it does remove your content from the training corpus, so the model is less likely to know your product exists well enough to go looking for it.

Keep reading

robots.txt for AI crawlers: a directive-by-directive reference
Every AI user agent worth naming, what each one controls, the precedence rules that decide which directive wins, and the wildcard mistake that quietly removes sites from AI answers.

See which answer engines already read this page

Shiftrank records every AI crawler hit from the moment your DNS resolves.

Start free