Robots Txt Generator

Robots.txt Generator

Generate a robots.txt file with Allow, Disallow rules and sitemap URL — copy-ready for SEO.

Your data never leaves your browser — all processing happens locally on your device.

About this tool

The Robots.txt Generator builds a standards-compliant robots.txt for search engine crawlers. Set user-agent, allow/disallow paths, optional crawl-delay and your sitemap URL. Copy the file to your site root to guide Google, Bing and other bots. Generated locally in your browser.

How to use the Robots.txt Generator

  1. 1

    Set crawler rules

    Enter user-agent, allow and disallow paths.

  2. 2

    Add sitemap URL

    Paste your XML sitemap URL (optional crawl-delay too).

  3. 3

    Copy robots.txt

    Copy the file and upload it to your website root.

Frequently Asked Questions

Where do I put robots.txt?

Upload it to your site root so it is available at https://yoursite.com/robots.txt.

What is Disallow?

Disallow tells crawlers not to fetch matching paths. Use Allow to override specific paths within a disallowed area.

Should I include a sitemap?

Yes. Adding your sitemap URL helps search engines discover all pages faster.

Related Tools

Meta Tag GeneratorGenerate SEO meta tags, Open Graph and Twitter Card HTML for any page — copy-ready in seconds.Word CounterCount words, characters, sentences and paragraphs with estimated reading time.Case ConverterConvert text to uppercase, lowercase, title case, camelCase, snake_case and more instantly.JSON FormatterFormat, minify and validate JSON online — pretty-print with syntax checking in your browser.