Robots.txt Generator

Create a custom robots.txt file with user-agent, allow, disallow, and sitemap directives for search engine crawlers.

User-agent Allow paths (one per line) Disallow paths (one per line) Sitemap URL

Generated robots.txt

This generator builds a valid robots.txt file for your site; copy it to your web root for crawler guidance.

Usage

Set a user-agent, allow public paths, and block private directories.
Add a sitemap URL to help crawlers find all indexed pages.
Copy the generated text to your site root as `robots.txt`.