Robots.txt Builder

Crawler rules, including opt-outs for the major AI crawlers.

Starting pointDisallow (one path per line)Allow (optional, one per line)

Sitemap URLCrawl-delay

Block AI training crawlers (GPTBot, ClaudeBot, PerplexityBot, CCBot, Google-Extended)

robots.txt

User-agent: *
Disallow: /admin
Disallow: /dashboard
Disallow: /api/
Disallow: /cart
Disallow: /checkout
Disallow: /*?*sessionid=