Generators100% private

Robots.txt Generator

Generate a valid robots.txt file with a visual rule builder.

User-agent

Disallow

Allow

Sitemap URL

User-agent: *
Disallow: /admin/
Disallow: /private/
Allow: /

Sitemap: https://example.com/sitemap.xml

How it works

1
Configure rulesSet user-agent patterns, allowed paths, and disallowed paths.
2
Add sitemapOptionally include your sitemap URL for search engines.
3
DownloadCopy or download the robots.txt file and place it in your site root.

What does robots.txt do?

It tells search engine crawlers which pages to crawl and which to skip. It's a suggestion - well-behaved bots follow it, but malicious ones may not.

What does 'User-agent: *' mean?

It applies the rules to all crawlers. You can target specific bots like 'Googlebot' with separate rules.

Where do I put this file?

Place it in the root of your website (e.g., https://example.com/robots.txt). Search engines look for it there automatically.

Generate Open Graph, Twitter Card, and standard meta tags for any URL.

Create linear, radial, and conic CSS gradients with a visual editor.

Generate cryptographically secure passwords with customizable length and character sets.