DevDungeon / WebGenome

A breadth first web crawler that stores HTTP headers in a MongoDB database with a web front end all written in Go.
25Updated 6 years ago

Alternatives and similar repositories for WebGenome:

Users that are interested in WebGenome are comparing it to the libraries listed below