Quicktil

network

Robots.txt Tester

Fetch and evaluate a robots.txt rule for a selected user-agent and path.

This check requires our server to fetch one public robots.txt file. Robots.txt is guidance for crawlers, not access control.

Keep exploring

Related tools

network

Sitemap Viewer

View entries and last modification dates from a public XML sitemap or sitemap index.

network

HTTP Status Checker

Check the HTTP status, response time, and redirects for a public URL.

network

Domain Extractor

Extract hostnames from text, URLs, and domain lists directly in your browser.

network

DNS Lookup

Query common DNS record types for a public hostname from our server.