Alternatives and similar repositories for crawl-anywhere

Users that are interested in crawl-anywhere are comparing it to the libraries listed below

• BayanGroup / nutch-custom-search
  ☆66Updated 8 years ago
• spinscale / elasticsearch-opennlp-plugin
  Additional opennlp mapping type for elasticsearch in order to perform named entity recognition
  ☆136Updated 9 years ago
• commoncrawl / commoncrawl-crawler
  The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)
  ☆219Updated 2 years ago
• jiminoc / goose
  Html Content / Article Extractor in Scala - open sourced from Gravity Labs - http://gravity.com
  ☆343Updated 6 years ago
• codelibs / elasticsearch-river-web
  Web Crawler for Elasticsearch
  ☆234Updated 5 years ago
• hollingsworthd / ScreenSlicer
  Automatic, zero-config web scraping -- written in Java, has no dependency on Java EE or app servers, and the web scraper has a restful/JS…
  ☆155Updated 8 years ago
• jprante / elasticsearch-langdetect
  A plugin for language detection in Elasticsearch using Nakatani Shuyo's language detector
  ☆252Updated 7 years ago
• pandastrike / bayzee
  Text classification using Naive Bayes and Elasticsearch
  ☆154Updated 9 years ago
• Wikidata / primarysources
  Approve or reject statements from third-party datasets
  ☆146Updated 7 years ago
• rufuspollock-okfn / facetview
  FacetView is a pure javascript frontend for ElasticSearch.
  ☆291Updated 10 years ago
• jprante / elasticsearch-index-termlist
  Elasticsearch Index Termlist
  ☆118Updated 6 years ago
• scrapinghub / mdr
  A python library detect and extract listing data from HTML page.
  ☆108Updated 8 years ago
• carrot2 / elasticsearch-carrot2
  Carrot2 plugin for ElasticSearch
  ☆291Updated 2 years ago
• nytlabs / hive
  A platform for backing crowdsourcing websites, built in golang for elasticsearch
  ☆359Updated 4 years ago
• Norconex / crawlers
  Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or fi…
  ☆194Updated last week
• derryx / elasticsearch-changes-plugin
  An Elasticsearch Plugin that notifies about changes to indices
  ☆92Updated 9 years ago
• DigitalPebble / behemoth
  Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.
  ☆283Updated 7 years ago
• CottageLabs / facetview2
  a pure javascript frontend for ElasticSearch search indices.
  ☆80Updated 7 years ago
• jaeksoft / opensearchserver
  Open-source Enterprise Grade Search Engine Software
  ☆509Updated 3 years ago
• lumifyio / lumify
  open source big data integration, analytics, and visualization
  ☆421Updated 8 years ago
• Parsely / serpextract
  Easy extraction of keywords and engines from search engine results pages (SERPs).
  ☆90Updated 3 years ago
• superfeedr / feediscovery
  Feed discovery to share :)
  ☆41Updated 8 years ago
• adamhub / screenbird
  a full cross platform video screen capture tool and host. Java based screen recorder, and Django based web backend. Also included is a di…
  ☆106Updated 9 years ago
• fergiemcdowall / solrstrap
  Solrstrap is a Query-Result interface for Solr written in JavaScript, HTML and CSS
  ☆87Updated 8 years ago
• mohaps / xtractor
  XTractor is an algorithmic text extractor from web pages written in Java. It builds upon the "commonly used web design practices" approac…
  ☆43Updated 9 years ago
• senseidb / sensei
  distributed realtime searchable database
  ☆117Updated 11 years ago
• openpreserve / pagelyzer
  Suite of tools for detecting changes in web pages and their rendering
  ☆55Updated last year
• SocialHarvest / dashboard
  A modular dashboard that visualizes data gathered by Social Harvest server (as well as from other sources).
  ☆36Updated 9 years ago
• OlivierBlanvillain / crawler
  Blog crawler for the blogforever project.
  ☆23Updated 11 years ago
• svola / ElasticUI-extension
  An extension to the demo template of ElasticUI a beautiful AngularJS frontend to ElasticSearch for faceted navigation
  ☆39Updated 10 years ago