The short answer
GPTBot is OpenAI's web crawler for collecting content that may be used to train its AI models. It is separate from OAI-SearchBot, which builds the index for ChatGPT search. ChatGPT-User is separate too; it fetches pages when a user asks. Blocking GPTBot does not remove your site from ChatGPT search results.
- GPTBot collects training data for OpenAI's models.
- OAI-SearchBot, not GPTBot, decides whether ChatGPT search can find and cite your pages.
- You can block GPTBot and still allow OAI-SearchBot.
- Blocking GPTBot affects future collection only, not content gathered earlier.
What does GPTBot do?
GPTBot visits public web pages and collects content. OpenAI says this content may be used to improve and train its models.
OpenAI states that GPTBot follows robots.txt rules. It also says it filters out some sources, such as sites that require payment to access.
GPTBot does not decide what ChatGPT says about you today. It affects what future models may learn from your pages.
Details change, so check OpenAI's official bot documentation for the current wording.
GPTBot vs OAI-SearchBot vs ChatGPT-User
OpenAI runs three separate agents. Mixing them up is the most common mistake site owners make.
Each one reads its own group in robots.txt. A rule for GPTBot says nothing about OAI-SearchBot, and the reverse is also true.
| User agent | Job | If you block it |
|---|---|---|
| GPTBot | Collects training data | Future content is not collected for training |
| OAI-SearchBot | Builds the ChatGPT search index | Your pages may not appear or be cited in ChatGPT search |
| ChatGPT-User | Fetches a page when a user asks | ChatGPT may not open your page during a conversation |
Should you block GPTBot?
This is a business decision, not a technical one. There is no right answer for everyone.
Think about what your pages contain. A local service business has little to lose from public pages being read. A publisher whose articles are the product may feel differently.
- Reasons to block: you sell content, you have licensing concerns, or you prefer not to contribute to training.
- Reasons to allow: you want future models to know your brand, services and facts.
- Either way, keep OAI-SearchBot allowed if you want to be found in ChatGPT search.
- Paid or members-only areas can be blocked on their own, without blocking the whole site.
How to allow or block GPTBot in robots.txt
This example blocks GPTBot from a members area only, and keeps ChatGPT search open. To block GPTBot everywhere, use Disallow: / instead.
User-agent: GPTBot Disallow: /members/ User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: /
OpenAI notes that robots.txt changes can take around a day to take effect. Check its docs for the current timing.
How to tell if GPTBot visits your site
Search your server or CDN logs for the text GPTBot in the user-agent field. Your host may also show bot traffic in a dashboard.
Visits tend to come in bursts rather than on a fixed schedule. A quiet week does not mean you are blocked.
Because user agents can be faked, OpenAI publishes the IP ranges its bots use. Compare suspicious visits against that list.
WebValet's free check reads your robots.txt and shows whether GPTBot, OAI-SearchBot and other AI bots are allowed.
Common GPTBot mistakes
These slip-ups are easy to make and easy to fix.
- Blocking GPTBot to hide from ChatGPT search, which it does not do.
- Using User-agent: * with Disallow: / and blocking every bot, including search engines.
- Assuming a block removes content that was collected before.
- Forgetting that a firewall or CDN rule may block OpenAI bots regardless of robots.txt.
What changes after you block or allow GPTBot?
Day to day, very little you can see. ChatGPT search keeps working as long as OAI-SearchBot is allowed.
The effect is on future training. If you block GPTBot, new content is not collected for training. Models trained later may know less about recent changes to your business.
If you allow it, your public pages may help shape what future models know. That is no promise of a mention. It is one input among many, alongside the wider web.
Questions people ask
Does blocking GPTBot remove my content from ChatGPT?
No. It stops future collection for training only. It does not affect ChatGPT search, which uses OAI-SearchBot, or content gathered earlier.
Does GPTBot respect robots.txt?
OpenAI says it does. If you see traffic that ignores your rules, check whether it truly comes from OpenAI's published IP ranges.
Is GPTBot the same as ChatGPT?
No. ChatGPT is the product people use. GPTBot is one of several crawlers OpenAI runs in the background.
Will allowing GPTBot make ChatGPT recommend me?
Not on its own. Being recommended depends on many signals, including third-party mentions. Nobody can guarantee it.
Should I block GPTBot on some pages only?
That is often a sensible middle ground. Keep public marketing pages open and block paid, private or members-only folders. Remember that pages behind a login are not reachable by crawlers anyway.