How to fix ai crawler blocked on BigCommerce
Your robots.txt explicitly blocks one or more AI assistants from crawling your store, so those engines can't discover, index, or cite your products when shoppers ask for recommendations.
Steps for BigCommerce
- Go to Channel Manager > your storefront > Customize robots.txt (or Settings > SEO depending on plan).
- Remove any 'Disallow: /' rule under the AI bot's user-agent, or add an explicit allow group for it.
- Save and verify at https://yourstore.com/robots.txt.
# Before (blocking ChatGPT's crawler):
User-agent: GPTBot
Disallow: /
# After (allowing it, while still protecting private paths):
User-agent: GPTBot
Allow: /
Disallow: /cart
Disallow: /checkout
Disallow: /accountWhat is ai crawler blocked?
robots.txt is a small text file at your domain root that tells automated crawlers which parts of your site they may read. A 'Disallow: /' rule under an AI crawler's user-agent (for example GPTBot for ChatGPT, or PerplexityBot for Perplexity) tells that assistant it may not read any page on your site.
AI assistants increasingly answer shopping queries by crawling and synthesizing content from stores they can access. A blocked crawler removes your store from consideration entirely for that engine - competitors who allow access get recommended in your place. Unblocking is a one-line change with outsized upside for AI visibility.
See the complete Ai crawler blocked guide for every platform and the full background.
Not sure if your BigCommerce store has this?
Run a free SEOLZ audit — we’ll find ai crawler blocked and every other issue across your whole site.
Scan my site free