Technical SEO generator
Robots.txt Generator
Create a robots.txt file with crawl rules, private-path blocks and a sitemap reference.
Generate robots.txt
Adjust the crawl settings and copy the generated file. The output updates directly from the options you choose.
Generator settings
Generated robots.txt
User-agent: * Allow: / Disallow: /admin/ Disallow: /login/ Disallow: /cart/ Disallow: /checkout/ Disallow: /wp-admin/ Sitemap: https://example.com/sitemap.xml
robots.txt should not be used to hide sensitive information. It only gives crawl instructions to search engines and does not protect private pages.
Deployment guidance
Upload this file to the root of your website so it appears at https://example.com/robots.txt. After publishing, check the URL in your browser and submit the sitemap in Google Search Console.
Free Website Growth Review
Not sure whether your site is being crawled properly?
Uppercut Labs can review your technical SEO setup and check whether search engines can properly access, understand, and index your website.
Book a Free Website Growth ReviewWhat it covers
- Crawler access rules
- Common admin and private paths
- Custom allow and disallow paths
- Sitemap reference and deployment guidance
Ready when you are
Bring us the idea.
Tell us what you want to launch, improve or automate. We will come back with a clear recommendation.