nadar / crawler

A Website Crawler Implementation written in PHP. High extendible, Indexes PDFs and is very memory efficient.
10Updated 2 weeks ago

Related projects

Alternatives and complementary repositories for crawler