scrapy / protegoLinks

A pure-Python robots.txt parser with support for modern conventions.
70Updated 2 weeks ago

Alternatives and similar repositories for protego

Users that are interested in protego are comparing it to the libraries listed below

Sorting: