Smerity / cc-warc-examplesLinks
CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop
☆56Updated 4 years ago
Alternatives and similar repositories for cc-warc-examples
Users that are interested in cc-warc-examples are comparing it to the libraries listed below
Sorting:
- Mirror of Apache Stanbol (incubating)☆115Updated last year
- Common web archive utility code.☆57Updated 3 weeks ago
- A set of reusable Java components that implement functionality common to any web crawler☆251Updated 2 weeks ago
- CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop☆37Updated last year
- Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.☆283Updated 7 years ago
- A text tagger based on Lucene / Solr, using FST technology☆177Updated 2 years ago
- Entity Extraction Text Processor☆148Updated 2 years ago
- Warcbase is an open-source platform for managing analyzing web archives☆161Updated 8 years ago
- Java library for reading and writing WARC files with a typed API☆52Updated this week
- Solr Dictionary Annotator (Microservice for Spark)☆71Updated 5 years ago
- Approve or reject statements from third-party datasets☆146Updated 7 years ago
- Solrstrap is a Query-Result interface for Solr written in JavaScript, HTML and CSS☆87Updated 8 years ago
- ☆185Updated 7 years ago
- The WikiBrain Java library enables researchers and developers to incorporate state-of-the-art Wikipedia-based algorithms and technologies…☆95Updated 7 years ago
- Launch AWS Elastic MapReduce jobs that process Common Crawl data.☆49Updated 8 years ago
- Additional opennlp mapping type for elasticsearch in order to perform named entity recognition☆136Updated 9 years ago
- The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)☆223Updated 3 years ago
- Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or fi…☆197Updated last week
- SemanticVectors creates semantic WordSpace models from free natural language text.☆220Updated 3 years ago
- WARC (Web Archive) Input and Output Formats for Hadoop☆37Updated 11 years ago
- Various tools for the DBpedia project - This does NOT contain the DBpedia extaction framework☆104Updated last year
- Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.☆419Updated 2 years ago
- Stanford CoreNLP NER addon for Apache Tika's NamerEntityParser☆13Updated 3 years ago
- Named-Entity Recognition extension for Google Refine / OpenRefine☆73Updated 8 years ago
- RDF store on a cloud-based architecture (previously on https://code.google.com/p/cumulusrdf)☆31Updated 9 years ago
- Search relevance evaluation toolkit☆74Updated 3 years ago
- Html Content / Article Extractor in Scala - open sourced from Gravity Labs - http://gravity.com☆343Updated 6 years ago
- SIREn - Semi-Structured Information Retrieval Engine☆108Updated 4 years ago
- Json Wikipedia, contains code to convert the Wikipedia xml dump into a json/avro dump☆255Updated 2 years ago
- SKOS Support for Apache Lucene and Solr☆56Updated 4 years ago