RobotsTesterBot and RobotsTxtChecker

robotstester.com fetches robots.txt files under two names. Both only ever request /robots.txt: they don't crawl pages, follow links or send forms.

When they visit

User agent	When it visits
`RobotsTxtChecker/1.0 (+https://robotstester.com/bot/)`	When someone checks your site with the checker, tester, validator or generator. One request each time.
`RobotsTesterBot/1.0 (+https://robotstester.com/bot/)`	About once a day for each site someone watches with email alerts, and again a few hours later if robots.txt didn't load.

How they fetch

They follow up to 5 redirects and read at most 500 KiB, as Google does.
They don't follow the rules in robots.txt, because robots.txt is the file those rules come from. Every crawler reads it before anything else.
Requests come from our host's servers, so the IP address can change.
If your server refuses them, the tools say robots.txt couldn't be read, and people watching your site are told once.

Questions or problems? Email alerts@robotstester.com.