AgentOK
Is your robots.txt blocking AI shopping assistants? How to tell
When a shopper asks ChatGPT, Claude or Perplexity for help buying something, the assistant has to read your store. If robots.txt forbids it, your store never comes up.
Three kinds of bots
- Assistants that read a page when a person asks, such as ChatGPT-User, Claude-User and Perplexity-User. They arrive when a shopper asks about your store. Blocking them means losing that demand.
- Search bots, such as OAI-SearchBot, Claude-SearchBot and PerplexityBot. They build the index that AI search draws on.
- Training bots, such as GPTBot and Google-Extended. They collect data for model training; blocking them does not decide whether an assistant can find you. That choice is yours.
How to check
Open yourstore.com/robots.txt and look for User-agent lines followed by Disallow: /. A common mistake is a blanket User-agent: * with Disallow: /, or a security plugin or CDN that blocks bots without you knowing.
What to do
Let assistants and search bots in. Allow or block training bots according to your business. Keep the basket, account and internal search closed to everyone.
AgentOK reads your robots.txt and shows, for each known bot, whether you let it in.