Alternatives and similar repositories for heritrix3:

Users that are interested in heritrix3 are comparing it to the libraries listed below

• apache / nutch
  Apache Nutch is an extensible and scalable web crawler
  ☆2,960Updated last week
• yasserg / crawler4j
  Open Source Web Crawler for Java
  ☆4,570Updated 3 years ago
• CrawlScript / WebCollector
  WebCollector is an open source web crawler framework based on Java.It provides some simple interfaces for crawling the Web,you can setup …
  ☆3,074Updated last week
• webrecorder / pywb
  Core Python Web Archiving Toolkit for replay and recording of web archives
  ☆1,434Updated 2 months ago
• internetarchive / brozzler
  brozzler - distributed browser-based web crawler
  ☆682Updated this week
• apache / incubator-stormcrawler
  A scalable, mature and versatile web crawler based on Apache Storm
  ☆896Updated this week
• xtuhcy / gecco
  Easy to use lightweight web crawler（易用的轻量化网络爬虫）
  ☆2,505Updated 10 months ago
• scrapy / scrapyd
  A service daemon to run Scrapy spiders
  ☆2,984Updated 3 weeks ago
• jaeksoft / opensearchserver
  Open-source Enterprise Grade Search Engine Software
  ☆503Updated 2 years ago
• code4craft / xsoup
  When jsoup meets XPath.
  ☆468Updated last year
• zhegexiaohuozi / SeimiCrawler
  一个简单、敏捷、分布式的支持SpringBoot的Java爬虫框架;An agile, distributed crawler framework.
  ☆1,980Updated last month
• scrapinghub / portia
  Visual scraping for Scrapy
  ☆9,338Updated 6 months ago
• Rhizome-Conifer / conifer
  Collect and revisit web pages.
  ☆1,490Updated last week
• code4craft / webmagic
  A scalable web crawler framework for Java.
  ☆11,473Updated 2 weeks ago
• scrapy-plugins / scrapy-splash
  Scrapy+Splash for JavaScript integration
  ☆3,171Updated last year
• scrapinghub / splash
  Lightweight, scriptable browser as a service with an HTTP API
  ☆4,112Updated 5 months ago
• apache / lucene-solr
  Apache Lucene and Solr open-source search software
  ☆4,371Updated 3 months ago
• iipc / awesome-web-archiving
  An Awesome List for getting started with web archiving
  ☆2,108Updated 2 weeks ago
• code4craft / jsoup-learning
  Jsoup学习笔记。添加了部分学习代码和注释。
  ☆638Updated last year
• qinxuye / cola
  A high-level distributed crawling framework.
  ☆1,502Updated 2 years ago
• gsh199449 / spider
  A configurable web spider with a easy-to-use web console
  ☆991Updated 6 years ago
• CrawlScript / nutcher
  nutcher是中文的nutch文档，包含nutch的配置和源码解析，持续更新中。
  ☆129Updated 5 years ago
• webrecorder / browsertrix-crawler
  Run a high-fidelity browser-based web archiving crawler in a single Docker container
  ☆686Updated this week
• machawk1 / wail
  Web Archiving Integration Layer: One-Click User Instigated Preservation
  ☆355Updated 3 months ago
• wycm / zhihu-crawler
  zhihu-crawler是一个基于Java的高性能、支持免费http代理池、支持横向扩展、分布式爬虫项目
  ☆914Updated 5 years ago
• ehcache / ehcache3
  Ehcache 3.x line
  ☆2,034Updated this week
• gigablast / open-source-search-engine
  Nov 20 2017 -- A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com…
  ☆1,545Updated last year
• apache / mahout
  Mirror of Apache Mahout
  ☆2,154Updated last month
• grangier / python-goose
  Html Content / Article Extractor, web scrapping lib in Python
  ☆3,998Updated 3 years ago
• YahooArchive / anthelion
  Anthelion is a plugin for Apache Nutch to crawl semantic annotations within HTML pages.
  ☆2,841Updated 9 years ago