utiloralab.
SEO tools / Free online tool

Robots.txt Generator

Create clear crawler rules from a list of paths, with an optional sitemap URL.

Runs in your browser
0 characters
OUTPUT
Your result will appear here

A little input. A useful output.

Your input stays on your device. No uploads. No account. Just your result.

How to use Robots.txt Generator

  1. 01

    Enter one disallowed path per line.

  2. 02

    Optionally add an absolute sitemap URL.

  3. 03

    Publish the result at the site root after reviewing the rules.

A quick example

Try a simple example

INPUT
/admin/
/api/
OUTPUT
User-agent: *
Disallow: /admin/
Disallow: /api/

About Robots.txt Generator

Robots.txt gives cooperating crawlers instructions about fetching paths. It is publicly visible and is not an authentication or index-removal mechanism.

When it comes in handy

  • Draft crawler exclusions.
  • Include a sitemap location.

Why use it here?

  • Runs locally in your browser.
  • No account needed; copy or download your result.

A few things to keep in mind

  • Disallow does not ensure a URL leaves the index.
  • Never use robots.txt to protect private files.

Good questions. Simple answers.

Can robots.txt hide private pages?

No. Enforce authentication for private pages. Crawlers that ignore robots rules can still request them.

Is my input uploaded?

No. This tool processes your input in your browser. Your input and output are not included in analytics.

A good next step

Useful tools that go well together.

Explore category