Guide · Squarespace

AI crawlers and llms.txt on Squarespace

Shaheer Malik
Shaheer Malik
Founder, WebValet
2 min readUpdated

The short answer

Squarespace does not let you edit robots.txt. Instead, Settings, then Crawlers, has a box to block known AI crawlers, unchecked by default. Leave it unchecked if you want AI answers to read you. On version 7.1 sites, write an llms.txt under SEO/AI Visibility, then SEO Settings, then the LLMS.txt tab.

  • Squarespace manages robots.txt for you; there is no editor.
  • The Block known artificial intelligence crawlers box is unchecked by default.
  • Checking it tells crawlers such as GPTBot, ClaudeBot, CCBot and Google-Extended not to crawl.
  • Version 7.1 sites can publish an llms.txt, which is off until you add text.

Which AI crawlers should you allow?

AI companies run separate crawlers for search and for training. Search crawlers decide whether AI answers can find, cite and link you.

Training crawlers collect content that may train future models. Blocking them does not remove you from AI search.

Keep allowed to be citedTraining crawlers, your choice
OAI-SearchBot (ChatGPT search)GPTBot (OpenAI)
Claude-SearchBot (Claude search)ClaudeBot (Anthropic)
PerplexityBot (Perplexity)CCBot (Common Crawl)
Googlebot and Bingbot (Google, Copilot)Google-Extended, Applebot-Extended (control tokens)

Where is the AI crawler setting on Squarespace?

Go to Settings, then Crawlers. The option is called Block known artificial intelligence crawlers.

Squarespace says checking it updates your robots.txt to ask listed bots not to crawl. The list includes GPTBot, ClaudeBot, CCBot, Google-Extended and others.

Squarespace also notes it cannot remove content already collected, and that robots.txt is a request some crawlers ignore.

After changing the box, open yourdomain.com/robots.txt and run the AI crawler checker to see exactly which crawlers it now blocks.

Can you allow some AI crawlers and block others?

Not with Squarespace's own settings. The single box covers its whole list, and Squarespace says you cannot limit AI crawlers to specific pages.

If you want search crawlers in but training crawlers out, the box is all or nothing. Decide which matters more to you: being cited, or staying out of training data.

How do you add llms.txt on Squarespace?

Squarespace's help center describes this for version 7.1 sites. The file is disabled by default, and you write the text yourself.

  1. 1

    Open the panel

    Open the SEO/AI Visibility panel and select SEO Settings.

  2. 2

    Open the tab

    Select the LLMS.txt tab.

  3. 3

    Add your text

    Paste your llms.txt text into the field. Squarespace points to llmstxt.org for the format.

  4. 4

    Save and check

    Select Save, then open yourdomain.com/llms.txt to confirm it loads.

Questions people ask

Does Squarespace block AI crawlers by default?

No. Squarespace says the Block known artificial intelligence crawlers box is unchecked by default. Check that nobody has ticked it.

Can I edit robots.txt on Squarespace?

Squarespace does not offer a robots.txt editor. The Crawlers settings are the supported way to change what AI crawlers are asked to do.

Does Squarespace write llms.txt for me?

No. Squarespace's llms.txt is a text field you fill in. Use an llms.txt generator to draft it, then paste it in.

Put it to work

Check it, track it, or have it done

No trial and no lock-in. Start free, then choose.

  1. 01 · Start here

    Free check

    $0no signup

    See how ready your site is for Google and AI crawlers. About 20 seconds.

    Run a free check
  2. 02 · Track it weekly

    Pro

    $19/month · or $209/year

    25 buyer prompts on ChatGPT, Gemini and Perplexity, against 3 competitors. No trial, cancel anytime.

    Get Pro
  3. 03 · Done for you

    Hire Team Shaheer Malik

    $249/month · Pro included

    Shaheer's team does your SEO and GEO every month: pages, citations, listings and fixes.

    Schedule a call