Alternatives and similar repositories for crawl-anywhere

Users that are interested in crawl-anywhere are comparing it to the libraries listed below

• hollingsworthd / ScreenSlicer
  Automatic, zero-config web scraping -- written in Java, has no dependency on Java EE or app servers, and the web scraper has a restful/JS…
  ☆155Updated 8 years ago
• spinscale / elasticsearch-opennlp-plugin
  Additional opennlp mapping type for elasticsearch in order to perform named entity recognition
  ☆136Updated 9 years ago
• commoncrawl / commoncrawl-crawler
  The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)
  ☆218Updated 2 years ago
• Norconex / crawlers
  Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or fi…
  ☆194Updated this week
• jprante / elasticsearch-langdetect
  A plugin for language detection in Elasticsearch using Nakatani Shuyo's language detector
  ☆252Updated 7 years ago
• BayanGroup / nutch-custom-search
  ☆66Updated 8 years ago
• jiminoc / goose
  Html Content / Article Extractor in Scala - open sourced from Gravity Labs - http://gravity.com
  ☆343Updated 6 years ago
• elastic / elasticsearch-river-twitter
  Twitter River Plugin for elasticsearch (STOPPED)
  ☆203Updated last year
• codelibs / elasticsearch-river-web
  Web Crawler for Elasticsearch
  ☆234Updated 5 years ago
• scrapinghub / mdr
  A python library detect and extract listing data from HTML page.
  ☆108Updated 8 years ago
• jaeksoft / opensearchserver
  Open-source Enterprise Grade Search Engine Software
  ☆509Updated 2 years ago
• SocialHarvest / dashboard
  A modular dashboard that visualizes data gathered by Social Harvest server (as well as from other sources).
  ☆36Updated 8 years ago
• lumifyio / lumify
  open source big data integration, analytics, and visualization
  ☆420Updated 8 years ago
• senseidb / sensei
  distributed realtime searchable database
  ☆117Updated 10 years ago
• carrot2 / elasticsearch-carrot2
  Carrot2 plugin for ElasticSearch
  ☆291Updated 2 years ago
• DigitalPebble / behemoth
  Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.
  ☆283Updated 7 years ago
• nytlabs / hive
  A platform for backing crowdsourcing websites, built in golang for elasticsearch
  ☆359Updated 4 years ago
• CottageLabs / facetview2
  a pure javascript frontend for ElasticSearch search indices.
  ☆80Updated 7 years ago
• searchdaimon / enterprise-search
  An open source search engine for corporate data and websites.
  ☆106Updated 8 years ago
• openpreserve / pagelyzer
  Suite of tools for detecting changes in web pages and their rendering
  ☆55Updated last year
• seagatesoft / sde
  Structured Data Extractor. An application to extract structured data from web pages. It uses Data Extraction Based on Partial Tree Alignm…
  ☆49Updated 13 years ago
• lucidworks / solrlogmanager
  solr-logstash
  ☆43Updated 9 years ago
• RazorFlow / framework
  The RazorFlow Dashboard Framework
  ☆133Updated 5 years ago
• spinscale / elasticsearch-facetgrapher
  Small hack to draw date histogram facets as graph using nvd3.js
  ☆54Updated 12 years ago
• mohaps / xtractor
  XTractor is an algorithmic text extractor from web pages written in Java. It builds upon the "commonly used web design practices" approac…
  ☆43Updated 9 years ago
• fergiemcdowall / solrstrap
  Solrstrap is a Query-Result interface for Solr written in JavaScript, HTML and CSS
  ☆87Updated 8 years ago
• sonian / elasticsearch-jetty
  ☆285Updated 4 years ago
• chrismattmann / imagecat
  ImageCat is an Apache OODT RADIX application that uses Apache Solr, Apache Tika and Apache OODT to ingest 10s of millions of files (image…
  ☆96Updated 6 years ago
• polyfractal / ElasticBayes
  Naive Bayes Classifier implemented with Elasticsearch Aggregations
  ☆51Updated 11 years ago
• evolvingweb / ajax-solr
  A JavaScript framework for creating user interfaces to Solr.
  ☆653Updated 3 years ago