Smerity / cc-warc-examplesLinks
CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop
☆57Updated 4 years ago
Alternatives and similar repositories for cc-warc-examples
Users that are interested in cc-warc-examples are comparing it to the libraries listed below
Sorting:
- Common web archive utility code.☆55Updated this week
- Warcbase is an open-source platform for managing analyzing web archives☆162Updated 7 years ago
- Mirror of Apache Stanbol (incubating)☆112Updated last year
- Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.☆282Updated 7 years ago
- SKOS Support for Apache Lucene and Solr☆56Updated 4 years ago
- The linked open dataset described at http://datahub.io/dataset/vu-wordnet, and the tools used to create it☆25Updated 4 years ago
- RDF-Centric Map/Reduce Framework and Freebase data conversion tool☆149Updated 3 years ago
- Automatic tagging and analysis of documents in an Apache Solr index for faceted search by RDF(S) Ontologies & SKOS thesauri☆47Updated 3 years ago
- The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)☆216Updated 2 years ago
- General Architecture for Text Engineering☆50Updated 9 years ago
- English Dependency Relationship Extractor☆85Updated 6 months ago
- Solr Dictionary Annotator (Microservice for Spark)☆71Updated 5 years ago
- Additional opennlp mapping type for elasticsearch in order to perform named entity recognition☆136Updated 9 years ago
- A set of reusable Java components that implement functionality common to any web crawler☆244Updated last week
- Entity Extraction Text Processor☆147Updated last year
- A text tagger based on Lucene / Solr, using FST technology☆176Updated last year
- Combines Apache OpenNLP and Apache Tika and provides facilities for automatically deriving sentiment from text.☆34Updated 2 years ago
- Named-Entity Recognition extension for Google Refine / OpenRefine☆72Updated 8 years ago
- an open-source data management platform for knowledge workers (https://github.com/dswarm/dswarm-documentation/wiki)☆54Updated 7 years ago
- Json Wikipedia, contains code to convert the Wikipedia xml dump into a json/avro dump☆253Updated last year
- SemanticVectors creates semantic WordSpace models from free natural language text.☆219Updated 2 years ago
- A queue-controlled browser automation tool for improving web crawl quality☆61Updated 4 months ago
- An open source toolkit for mining Wikipedia☆129Updated 6 years ago
- Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.☆46Updated 7 years ago
- An easy-to-use and highly customizable crawler that enables you to create your own little Web archives (WARC/CDX)☆25Updated 7 years ago
- Hadoop jobs for WikiReverse project. Parses Common Crawl data for links to Wikipedia articles.☆38Updated 6 years ago
- `Slib` is a JAVA library dedicated to semantic data mining based on texts and/or ontology processing. The library is composed of various …☆83Updated last year
- DKPro C4CorpusTools is a collection of tools for processing CommonCrawl corpus, including Creative Commons license detection, boilerplate…☆52Updated 5 years ago
- Approve or reject statements from third-party datasets☆146Updated 7 years ago
- NEWS: JATE2.0 Beta.11 Released, see details below.☆81Updated last year