vodafon / waybackrobots

Returns disallowed paths from robots.txt found on your target domain and snapshotted by the Wayback Machine
26Updated 2 years ago

Related projects

Alternatives and complementary repositories for waybackrobots