Alternatives and similar repositories for commoncrawl-crawler:

Users that are interested in commoncrawl-crawler are comparing it to the libraries listed below

• jprante / elasticsearch-index-termlist
  Elasticsearch Index Termlist
  ☆117Updated 5 years ago
• LinkedInAttic / rest.li-api-hub
  API Hub is a web UI for browsing and searching a catalog of Rest.li APIs.
  ☆74Updated 5 years ago
• spinscale / elasticsearch-opennlp-plugin
  Additional opennlp mapping type for elasticsearch in order to perform named entity recognition
  ☆136Updated 8 years ago
• DigitalPebble / behemoth
  Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.
  ☆281Updated 6 years ago
• knowitall / openregex
  An efficient and flexible token-based regular expression language and engine.
  ☆75Updated 10 years ago
• apache / gora
  The Apache Gora open source framework provides an in-memory data model and persistence for big data.
  ☆120Updated 11 months ago
• senseidb / sensei
  distributed realtime searchable database
  ☆116Updated 10 years ago
• akkumar / hbasene
  HBase as the backing store for the TF-IDF representations for Lucene
  ☆108Updated 14 years ago
• thunderain-project / thunderain
  A Real-Time Analytical Processing (RTAP) example using Spark/Shark
  ☆51Updated 10 years ago
• jprante / elasticsearch-skywalker
  Skywalker for Elasticsearch is like Luke for Lucene
  ☆79Updated 4 years ago
• elsevierlabs-os / soda
  Solr Dictionary Annotator (Microservice for Spark)
  ☆71Updated 4 years ago
• emorynlp / nlp4j-old
  NLP tools developed by Emory University.
  ☆60Updated 8 years ago
• bixo / bixo
  Bixo is an open source web mining toolkit that runs as a series of Cascading pipes on top of Hadoop. By building a customized Cascading p…
  ☆142Updated 2 years ago
• Smerity / cc-warc-examples
  CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop
  ☆56Updated 3 years ago
• lucidworks / searchhub
  Fusion demo app searching open-source project data from the Apache Software Foundation
  ☆42Updated 6 years ago
• lucidworks / auto-phrase-tokenfilter
  Lucene Auto Phrase TokenFilter implementation
  ☆59Updated 6 years ago
• pranab / hoidla
  Set of real time stream processing algorithms that can be used by big data streaming platform
  ☆72Updated 4 years ago
• rdelbru / SIREn
  SIREn - Semi-Structured Information Retrieval Engine
  ☆107Updated 3 years ago
• basis-technology-corp / Java-readability
  A port of the arclabs 'readability' package to Java
  ☆72Updated 12 years ago
• sgroschupf / katta
  Katta - distributed Lucene
  ☆60Updated 11 years ago
• RedisLabs / spark-timeseries
  A library for financial and time series calculations on Apache Spark
  ☆28Updated 8 years ago
• o19s / lazy-semantic-indexing
  Elasticsearch Latent Semantic Indexing experimentation
  ☆33Updated 5 years ago
• Findwise / Hydra
  Distributed processing framework for search solutions
  ☆81Updated 2 years ago
• lucidworks / query-autofiltering-component
  A Query Autofiltering SearchComponent for Solr that can translate free-text queries into structured queries using index metadata
  ☆28Updated 6 years ago
• javasoze / clue
  command line tool for Apache Lucene
  ☆160Updated 5 months ago
• ept / warc-hadoop
  WARC (Web Archive) Input and Output Formats for Hadoop
  ☆35Updated 10 years ago
• stepthom / lucene-lda
  Using latent Dirichlet allocation (LDA) in Apache Lucene
  ☆58Updated 12 years ago
• ClearTK / cleartk
  Machine learning components for Apache UIMA
  ☆129Updated last year
• codelibs / elasticsearch-minhash
  Elasticsearch plugin for b-bit minhash algorism
  ☆62Updated 7 months ago
• grmblfrz / elasticsearch-zookeeper
  ☆40Updated 9 years ago