Free tool · no signup
robots.txt Checker
Parse any site's robots.txt and see what each AI crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Bingbot) is allowed or blocked from doing. Catches common mistakes — missing file, syntax errors, accidentally blocking AI search.
See the full picture — not just robots.txt
The Nova Insight Website Audit checks AI crawler access alongside 300+ other technical factors — schema markup, structured data, page-speed, security headers — and delivers a branded PDF with prioritised fixes.
