Alternatives and similar repositories for incubator-stormcrawler:

Users that are interested in incubator-stormcrawler are comparing it to the libraries listed below

• USCDataScience / sparkler
  Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
  ☆413Updated 2 years ago
• crawler-commons / crawler-commons
  A set of reusable Java components that implement functionality common to any web crawler
  ☆243Updated last week
• apache / nutch
  Apache Nutch is an extensible and scalable web crawler
  ☆3,000Updated this week
• elastic / elasticsearch-hadoop
  Elasticsearch real-time search and analytics natively integrated with Hadoop
  ☆1,935Updated this week
• VIDA-NYU / ache
  ACHE is a web crawler for domain-specific search.
  ☆464Updated last year
• OryxProject / oryx
  Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
  ☆1,782Updated 3 years ago
• strapdata / elassandra
  Elassandra = Elasticsearch + Apache Cassandra
  ☆1,714Updated last year
• apache / gobblin
  A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, orga…
  ☆2,238Updated last week
• lucidworks / banana
  Banana for Solr - A Port of Kibana
  ☆670Updated 7 months ago
• rayokota / hgraphdb
  HBase as a TinkerPop Graph Database
  ☆256Updated last week
• apache / datasketches-java
  A software library of stochastic streaming algorithms, a.k.a. sketches.
  ☆906Updated this week
• lucidworks / spark-solr
  Tools for reading data from Solr as a Spark RDD and indexing objects from Spark into Solr using SolrJ.
  ☆446Updated last year
• kohlschutter / boilerpipe
  Work in progress transmit from Google Code
  ☆1,114Updated 7 years ago
• o19s / solr-to-es
  Migrate a Solr node to an Elasticsearch index.
  ☆55Updated last year
• commoncrawl / commoncrawl-crawler
  The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)
  ☆213Updated 2 years ago
• apache / samza
  Mirror of Apache Samza
  ☆824Updated 3 weeks ago
• carrot2 / carrot2
  Carrot2: Text Clustering Algorithms and Applications
  ☆799Updated last month
• optimaize / language-detector
  Language Detection Library for Java
  ☆575Updated 2 years ago
• jayzeng / scrapy-elasticsearch
  A scrapy pipeline which send items to Elastic Search server
  ☆328Updated 2 years ago
• linkedin / URL-Detector
  A Java library to detect and normalize URLs in text
  ☆783Updated 2 years ago
• LinkedInAttic / camus
  LinkedIn's previous generation Kafka to HDFS pipeline.
  ☆876Updated 4 years ago
• actionml / universal-recommender
  Highly configurable recommender based on PredictionIO and Mahout's Correlated Cross-Occurrence algorithm
  ☆671Updated 5 years ago
• druid-io / tranquility
  Tranquility helps you send real-time event streams to Druid and handles partitioning, replication, service discovery, and schema rollover…
  ☆516Updated 5 years ago
• google / openrtb
  OpenRTB model for Java and other languages via protobuf; Helper OpenRTB libraries for Java including JSON serialization
  ☆400Updated last year
• apache / drill
  Apache Drill is a distributed MPP query layer for self describing data
  ☆1,963Updated 2 weeks ago
• commoncrawl / news-crawl
  News crawling with StormCrawler - stores content as WARC
  ☆339Updated last month
• Norconex / crawlers
  Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or fi…
  ☆187Updated this week
• Netflix / genie
  Distributed Big Data Orchestration Service
  ☆1,725Updated last week
• Netflix / suro
  Netflix's distributed Data Pipeline
  ☆793Updated last year
• spinscale / elasticsearch-ingest-opennlp
  An Elasticsearch ingest processor to do named entity extraction using Apache OpenNLP
  ☆270Updated 2 years ago