Alternatives and similar repositories for commoncrawl-crawler

Users that are interested in commoncrawl-crawler are comparing it to the libraries listed below

• Smerity / cc-warc-examples
  CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop
  ☆56Updated 4 years ago
• pranab / hoidla
  Set of real time stream processing algorithms that can be used by big data streaming platform
  ☆72Updated 4 years ago
• apache / gora
  The Apache Gora open source framework provides an in-memory data model and persistence for big data.
  ☆121Updated last year
• sgroschupf / katta
  Katta - distributed Lucene
  ☆60Updated 12 years ago
• lucidworks / storm-solr
  Storm / Solr Integration
  ☆19Updated last year
• akkumar / hbasene
  HBase as the backing store for the TF-IDF representations for Lucene
  ☆108Updated 15 years ago
• Heliosearch / heliosearch
  The next generation of open source search
  ☆92Updated 8 years ago
• DigitalPebble / TextClassification
  A Text Classification API in Java originally developed by DigitalPebble Ltd. The API is independent from the ML implementations used and …
  ☆48Updated 3 years ago
• spinscale / elasticsearch-opennlp-plugin
  Additional opennlp mapping type for elasticsearch in order to perform named entity recognition
  ☆136Updated 9 years ago
• kijiproject / kiji
  The Kiji project suite
  ☆33Updated 9 years ago
• thunderain-project / thunderain
  A Real-Time Analytical Processing (RTAP) example using Spark/Shark
  ☆51Updated 11 years ago
• elsevierlabs-os / soda
  Solr Dictionary Annotator (Microservice for Spark)
  ☆71Updated 5 years ago
• Findwise / Hydra
  Distributed processing framework for search solutions
  ☆81Updated 2 years ago
• OpenSextant / SolrTextTagger
  A text tagger based on Lucene / Solr, using FST technology
  ☆176Updated last year
• LinkedInAttic / rest.li-api-hub
  API Hub is a web UI for browsing and searching a catalog of Rest.li APIs.
  ☆74Updated 6 years ago
• YahooArchive / samoa
  SAMOA (Scalable Advanced Massive Online Analysis) is an open-source platform for mining big data streams.
  ☆425Updated 9 years ago
• momer / nutch-selenium
  ☆28Updated 8 years ago
• senseidb / sensei
  distributed realtime searchable database
  ☆117Updated 10 years ago
• calrissian / flowmix
  Flowmix is a flexible event processing engine for Apache Storm. It supports complex correlations of events via sliding/tumbling windows. …
  ☆58Updated 9 years ago
• DigitalPebble / behemoth
  Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.
  ☆282Updated 7 years ago
• adobe-research / spindle
  Next-generation web analytics processing with Scala, Spark, and Parquet.
  ☆331Updated 10 years ago
• apache / lens
  Mirror of Apache Lens
  ☆60Updated 5 years ago
• stefansavev / elasticsearch-custom-similarity-example
  An example how to implement a custom similarity (overlap similarity) for elasticsearch
  ☆41Updated 9 years ago
• lumongo / lumongo
  Distributed Realtime Search with Lucene and MongoDB
  ☆59Updated 7 years ago
• RedisLabs / spark-timeseries
  A library for financial and time series calculations on Apache Spark
  ☆28Updated 9 years ago
• gridgain / gridgain-old
  ☆264Updated 9 years ago
• bytefish / ElasticUtils
  Provides simplified access to the ElasticSearch Java API.
  ☆4Updated 4 years ago
• codelibs / elasticsearch-minhash
  Elasticsearch plugin for b-bit minhash algorism
  ☆63Updated 11 months ago
• P7h / StormTweetsSentimentAnalysis
  Computes sentiment analysis of tweets of US States in real-time using Storm.
  ☆62Updated 8 years ago
• crawler-commons / crawler-commons
  A set of reusable Java components that implement functionality common to any web crawler
  ☆243Updated last week