USCDataScience / sparklerLinks
Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
☆418Updated 2 years ago
Alternatives and similar repositories for sparkler
Users that are interested in sparkler are comparing it to the libraries listed below
Sorting:
- A scalable, mature and versatile web crawler based on Apache Storm☆946Updated last week
- Tools for reading data from Solr as a Spark RDD and indexing objects from Spark into Solr using SolrJ.☆445Updated 2 months ago
- Spark RDD with Lucene's query and entity linkage capabilities☆128Updated 2 months ago
- An Elasticsearch ingest processor to do named entity extraction using Apache OpenNLP☆274Updated 3 years ago
- A text tagger based on Lucene / Solr, using FST technology☆177Updated last year
- Score documents with pure dot product / cosine similarity with ES☆254Updated 4 years ago
- Query preprocessor for Java-based search engines (Querqy Core and Lucene implementation)☆188Updated this week
- The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)☆221Updated 2 years ago
- The Apache Gora open source framework provides an in-memory data model and persistence for big data.☆121Updated last year
- This project combines Apache Spark and Elasticsearch to enable mining & prediction for Elasticsearch.☆211Updated 11 years ago
- A set of reusable Java components that implement functionality common to any web crawler☆246Updated last week
- Apache Spark applications☆70Updated 7 years ago
- Streaming tweets with spark, language detection & sentiment analysis, dashboard with Kibana☆104Updated 9 years ago
- Scalable query engine for web scrapping/data mashup/acceptance QA, powered by Apache Spark☆142Updated last month
- Dice Solr Plugins from Simon Hughes Dice.com☆88Updated 4 years ago
- Code to index HDFS to Solr using MapReduce☆52Updated 6 years ago
- Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or fi…☆196Updated this week
- Elastic Search on Spark☆112Updated 11 years ago
- Serverless proxy for Spark cluster☆324Updated 5 years ago
- Carrot2 plugin for ElasticSearch☆291Updated 2 years ago
- Mirror of Apache Atlas (Incubating)☆95Updated 2 years ago
- Additional opennlp mapping type for elasticsearch in order to perform named entity recognition☆136Updated 9 years ago
- Elasticsearch entity resolution plugin based on Duke☆209Updated 5 years ago
- HBase as a TinkerPop Graph Database☆261Updated this week
- Continuous scalable web crawler built on top of Flink and crawler-commons☆52Updated 6 years ago
- spark + drools☆103Updated 3 years ago
- Divolte Collector☆282Updated 4 years ago
- Kite SDK☆393Updated 3 years ago
- A simple Spark LDA example. to demonstrate a full fletched clustering algorithm, with data cleaning using the processess like lemmatizati…☆23Updated 9 years ago
- Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in…☆1,036Updated 2 years ago