p0dalirius / robotstester

This Python script can enumerate all URLs present in robots.txt files, and test whether they can be accessed or not.
39Updated 11 months ago

Related projects: