jimsmart / grobotstxt

grobotstxt is a native Go port of Google's robots.txt parser and matcher library.
108Updated 2 years ago

Alternatives and similar repositories for grobotstxt:

Users that are interested in grobotstxt are comparing it to the libraries listed below