Alternatives and similar repositories for stormcrawler

Users that are interested in stormcrawler are comparing it to the libraries listed below

• USCDataScience / sparkler
  Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
  ☆418Updated 2 years ago
• crawler-commons / crawler-commons
  A set of reusable Java components that implement functionality common to any web crawler
  ☆247Updated last week
• Norconex / crawlers
  Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or fi…
  ☆196Updated last week
• scrapinghub / frontera
  A scalable frontier for web crawlers
  ☆1,320Updated 5 months ago
• kohlschutter / boilerpipe
  Work in progress transmit from Google Code
  ☆1,125Updated 7 years ago
• lucidworks / banana
  Banana for Solr - A Port of Kibana
  ☆672Updated 3 months ago
• apache / nutch
  Apache Nutch is an extensible and scalable web crawler
  ☆3,089Updated last week
• istresearch / scrapy-cluster
  This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
  ☆1,219Updated 2 years ago
• crawljax / crawljax
  Crawljax
  ☆535Updated 2 years ago
• commoncrawl / commoncrawl-crawler
  The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)
  ☆221Updated 2 years ago
• carrot2 / carrot2
  Carrot2: Text Clustering Algorithms and Applications
  ☆835Updated 3 weeks ago
• dragnet-org / dragnet
  Just the facts -- web page content extraction
  ☆1,274Updated 4 months ago
• scrapy / scrapely
  A pure-python HTML screen-scraping library
  ☆1,887Updated 3 years ago
• scrapinghub / extruct
  Extract embedded metadata from HTML markup
  ☆935Updated last month
• jiminoc / goose
  Html Content / Article Extractor in Scala - open sourced from Gravity Labs - http://gravity.com
  ☆343Updated 6 years ago
• internetarchive / heritrix3
  Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
  ☆3,089Updated 2 weeks ago
• momer / nutch-selenium
  ☆28Updated 9 years ago
• optimaize / language-detector
  Language Detection Library for Java
  ☆585Updated 3 years ago
• spinscale / elasticsearch-ingest-opennlp
  An Elasticsearch ingest processor to do named entity extraction using Apache OpenNLP
  ☆274Updated 3 years ago
• apache / gora
  The Apache Gora open source framework provides an in-memory data model and persistence for big data.
  ☆121Updated last year
• karussell / snacktory
  Readability clone in Java
  ☆460Updated 5 years ago
• datumbox / datumbox-framework
  Datumbox is an open-source Machine Learning framework written in Java which allows the rapid development of Machine Learning and Statisti…
  ☆1,087Updated last year
• apache / opennlp
  Apache OpenNLP
  ☆1,556Updated last week
• querqy / querqy
  Query preprocessor for Java-based search engines (Querqy Core and Lucene implementation)
  ☆188Updated last week
• OryxProject / oryx
  Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
  ☆1,784Updated 4 years ago
• carrot2 / elasticsearch-carrot2
  Carrot2 plugin for ElasticSearch
  ☆291Updated 2 years ago
• lumifyio / lumify
  open source big data integration, analytics, and visualization
  ☆419Updated 8 years ago
• espertechinc / esper
  Esper Complex Event Processing, Streaming SQL and Event Series Analysis
  ☆870Updated last year
• ikreymer / cdx-index-client
  A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/
  ☆205Updated 7 years ago
• elastic / elasticsearch-mapper-attachments
  Mapper Attachments Type plugin for Elasticsearch
  ☆505Updated 2 years ago