ept / warc-hadoopLinks
WARC (Web Archive) Input and Output Formats for Hadoop
☆37Updated 10 years ago
Alternatives and similar repositories for warc-hadoop
Users that are interested in warc-hadoop are comparing it to the libraries listed below
Sorting:
- Puck is a lightning-fast parser for natural languages using GPUs☆250Updated 10 years ago
- Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.☆283Updated 7 years ago
- Apache OpenNLP Sandbox☆44Updated this week
- The Cognitive Foundry is an open-source Java library for building intelligent systems using machine learning☆134Updated 4 years ago
- The Sweble Wikitext Components module provides a parser for MediaWiki's wikitext and an engine trying to emulate the behavior of a MediaW…☆73Updated last year
- Educational Examle of a custom Lucene Query & Scorer☆48Updated 5 years ago
- The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)☆220Updated 2 years ago
- NLP tools developed by Emory University.☆61Updated 9 years ago
- RDF-Centric Map/Reduce Framework and Freebase data conversion tool☆149Updated 3 years ago
- ☆41Updated 8 years ago
- Solr Dictionary Annotator (Microservice for Spark)☆71Updated 5 years ago
- Spark implementation of the Google Correlate algorithm to quickly find highly correlated vectors in huge datasets☆93Updated 9 years ago
- CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop☆37Updated 9 months ago
- A CPU and GPU-accelerated matrix library for data mining☆266Updated 4 years ago
- ☆92Updated 9 years ago
- Combines Apache OpenNLP and Apache Tika and provides facilities for automatically deriving sentiment from text.☆34Updated 2 years ago
- Machine learning components for Apache UIMA☆131Updated 2 years ago
- Chalk is a natural language processing library.☆260Updated 8 years ago
- NLP framework for JVM languages.☆151Updated 4 years ago
- Machine Learning for Cascading☆83Updated 10 years ago
- The WikiBrain Java library enables researchers and developers to incorporate state-of-the-art Wikipedia-based algorithms and technologies…☆95Updated 7 years ago
- Hadoop jobs for WikiReverse project. Parses Common Crawl data for links to Wikipedia articles.☆38Updated 7 years ago
- Big Data Science Swiss Army Knife - http://www.tuktu.io --☆61Updated 7 years ago
- An example of running Apache Spark using Scala in ipython notebook☆140Updated 10 years ago
- NLP toolkit (tokenizer, POS-tagger, parser, etc.)☆43Updated 8 years ago
- Scala utilities for teaching computational linguistics and prototyping algorithms.☆42Updated 12 years ago
- Apache Joshua☆109Updated 5 years ago
- Course repository for Applied Natural Language Processing☆125Updated 12 years ago
- A library of examples showing how to use the Common Crawl corpus (2008-2012, ARC format)☆65Updated 9 years ago
- A language detection library for the JVM☆36Updated 2 years ago