Robots.txt Tester
Is a URL blocked or allowed for Google?
Runs in your browser Instant results 100% free
Loading tool…
About the Robots.txt Tester
One wrong line in robots.txt can hide important pages from Google. Paste your robots.txt, enter a URL and choose a bot to see whether the URL is allowed or blocked and exactly which rule decided it – using the same longest-match logic Google uses.
How to use it
- Paste the content of your robots.txt (yourdomain.com/robots.txt).
- Enter the URL or path to test and choose the bot.
- Read the result and the matching rule, then fix the file if needed.
Frequently asked questions
How does Google choose between Allow and Disallow?
The most specific (longest) matching rule wins. If they are equally long, Allow wins.
Does Disallow remove a page from Google?
No. It stops crawling, but a blocked URL can still appear in results. Use a noindex tag to keep a page out.
Want to master this skill?Learn it step by step in our SEO course – practical, in Hindi and English.
See the course
