Robots.txt Analyzer

PKCapra SEO Tools

Free Robots.txt Analyzer

Fetch, parse and test a website's robots.txt with crawler-specific access simulation and rule explanations.

Public robots.txt only. The access simulator explains which parsed rule wins for each crawler profile.