Robots.txt Generator
Build a valid robots.txt with per-user-agent Allow/Disallow rules, crawl-delay and sitemap directives. Use a preset to allow all, block all, block AI crawlers or apply WordPress defaults — then copy or download. Everything runs locally in your browser — nothing is uploaded.
About robots.txt
Each group targets a crawler (* = all) with its own Allow / Disallow paths.
An empty Disallow: permits everything; Disallow: / blocks the whole site.
One click disallows GPTBot, CCBot, Google-Extended, ClaudeBot, PerplexityBot and more.
List absolute sitemap URLs so search engines can find your pages faster.
Save the file as robots.txt at your site root: https://example.com/robots.txt.
Well-behaved bots obey robots.txt, but it isn't a security control — don't rely on it to hide private pages.