Robots.txt Generator
SEO Tool
Robots.txt Generator
Create a perfect robots.txt file for your website in seconds. Control which search engines can crawl your pages — 100% free.
Website Details
Enter your website URL and sitemap location
Default Crawl Rule
Set the default rule for all search engine bots
✅ Allow All Bots
All search engines can crawl your entire website
🚫 Block All Bots
No search engine can crawl your website
⚙️ Custom Rules
Set specific rules for specific bots
Search Engine Bots
Choose which specific bots to allow or block
✅ Allow these bots
🚫 Block these bots
Block These Pages/Folders
Tell bots NOT to crawl these paths on your website
Common blocked paths: /admin/, /login/, /cart/, /checkout/, /private/, /wp-admin/ etc.
No blocked paths added yet.
Force Allow These Pages
Override block rules — always allow bots to crawl these paths
No forced-allow paths added yet.
robots.txt Output
LIVE
# Generated by Toolvix.xyz
# Fill in the form to generate your robots.txt
How to Use robots.txt
Step 1: Generate your robots.txt using this tool.
Step 2: Download the file or copy the content.
Step 3: Upload the file to your website’s root folder.
Step 4: Access it at: https://yoursite.com/robots.txt
Step 5: Test it in Google Search Console.
What is robots.txt?
A robots.txt file tells search engine crawlers which pages or files they can or cannot request from your website.
It helps protect private pages from being indexed on Google and improves your crawl budget.
⚠️ Note: robots.txt is a suggestion, not a rule. Malicious bots may still ignore it.
Copied!