Spix0r / robofinderLinks
Robofinder retrieves historical #robots.txt files from #Archive.org, allowing you to collect old directories and paths for any domain which can helps you in your #OSINT and #recon process.
☆222Updated last week
Alternatives and similar repositories for robofinder
Users that are interested in robofinder are comparing it to the libraries listed below
Sorting: