Site Settings
Crawler Access
Click any crawler to toggle between Allow and Block.
Quick Block Presets
Admin & CMS Paths
/admin/, /wp-admin/, /dashboard/, /login/
Private / Internal Pages
/private/, /internal/, /staging/
Search & Filter URLs
/?s=, /?filter=, /?sort=
Cart & Checkout
/cart/, /checkout/, /order-received/
Duplicate / Print Pages
/?print=1, /print/, /?format=pdf
API & JSON Endpoints
/api/, /wp-json/, /.json
Upload & Media Directories
/wp-content/uploads/, /assets/
Custom Rules
Live Preview
# Your robots.txt will appear here…
robots.txt Ready
File Location
Place the file at your site root:
yoursite.com/robots.txtNot a Security Tool
Robots.txt is a directive, not a block. Malicious bots may ignore it. Use server rules for real protection.
Cache Refresh
Google may cache your robots.txt for up to 24 hours before picking up changes.
Test It
Use Google Search Console → Robots.txt tester to validate your file after uploading.