The short answer
Claude, made by Anthropic, answers from its training data unless web search is used. With web search on, it looks up pages and cites them. Anthropic runs three bots: ClaudeBot for training, Claude-SearchBot for search, and Claude-User for user-requested fetches. Allow Claude-SearchBot and Claude-User if you want Claude to find and cite your pages.
- Claude answers from training data unless it searches the web.
- ClaudeBot collects training data; Claude-SearchBot and Claude-User support search and user requests.
- Many business tools are built on Claude, and they may use their own data sources.
- WebValet tracks Claude as a paid add-on, not in Pro by default.
How does Claude find information?
Claude's model learned from training data up to a cutoff date. Without search, it answers from that knowledge, which can be dated.
Claude can also search the web. Availability depends on the product, plan and settings, so check Anthropic's help pages. When it searches, it reads pages and cites them in the answer.
Anthropic separates its bots by purpose. Its documentation describes each one and how robots.txt applies.
| User agent | Purpose | If you block it |
|---|---|---|
| ClaudeBot | Collects content that may train Anthropic models | You opt out of future training |
| Claude-SearchBot | Crawls pages to improve Claude's search results | Your pages may be less visible in Claude's search |
| Claude-User | Fetches a page when a Claude user asks | Claude may not open your pages on request |
Claude is used in more places than its own app
Many companies build tools on Claude through Anthropic's API. Writing assistants, support bots and research tools may all run on it.
In those tools, answers may come from data the developer provides, not from web search. So how Claude describes you can vary widely by product.
That is another reason clear, consistent facts on your own site matter. They are what any tool is most likely to read.
If you sell to other businesses, some of your buyers may meet Claude inside their own work tools.
What does Claude tend to show?
With web search, Claude usually adds citations that link to the pages it used. Without search, it may name brands from memory with no links.
Claude often says when it is unsure or when information may be out of date. Still, check every fact it states about you. Behavior varies by version and settings.
Smaller or newer businesses may get a cautious reply, or none at all, without search. That usually reflects thin coverage in training data, not a penalty.
How to make your site readable for Claude
Allow the search and user bots if you want Claude to cite you. Decide separately about ClaudeBot and training.
Anthropic says its bots respect robots.txt. Some older rule sets name agents Anthropic no longer lists, so review those lines too.
- Check your firewall or CDN is not blocking Anthropic's bots by default.
- Keep key facts in plain HTML text, not only in images or scripts.
- Write an About page that states clearly who you are, what you do and where.
# Claude search User-agent: Claude-SearchBot Allow: / # Pages fetched when a Claude user asks User-agent: Claude-User Allow: / # Model training (optional opt-out) User-agent: ClaudeBot Disallow: /
How to check what Claude says about your brand
Use claude.ai and test both modes, because answers can differ a lot.
- 1
Ask without search
Ask a buyer question with web search off. This shows what the model knows from training.
- 2
Ask with search
Turn web search on and ask again. Note the cited pages and who is named.
- 3
Ask about your brand
Ask what Claude knows about your business. Check services, location and any claims it makes.
- 4
Record and repeat
Save the answers and check again in a few weeks to spot changes.
Does WebValet track Claude?
Claude tracking is a paid add-on to WebValet Pro, not included by default. Pro covers ChatGPT, Gemini and Perplexity. The add-on asks your tracked prompts on Claude as well.
WebValet also uses Claude to write its recommendations, grounded in your data. That is separate from tracking what Claude says about you.
Questions people ask
Does blocking ClaudeBot stop Claude from citing me?
Anthropic describes ClaudeBot as its training crawler. Search and user fetches use Claude-SearchBot and Claude-User. Check Anthropic's documentation for how each one is handled today.
Does Claude always search the web?
No. It depends on the product, plan and settings, and on whether the question needs fresh facts. Answers without search come from training data.
Is Claude included in WebValet Pro?
Not by default. Pro tracks ChatGPT, Gemini and Perplexity. Claude is available as a paid add-on.
Why does Claude say it does not know my business?
Without web search, Claude only knows what was in its training data. Smaller or newer businesses may not be covered. Try again with search on, and make sure Claude-SearchBot can reach your site.