Smerity / cc-warc-examplesLinks
CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop
☆57Updated 4 years ago
Alternatives and similar repositories for cc-warc-examples
Users that are interested in cc-warc-examples are comparing it to the libraries listed below
Sorting:
- Common web archive utility code.☆56Updated 2 months ago
- Warcbase is an open-source platform for managing analyzing web archives☆162Updated 7 years ago
- Mirror of Apache Stanbol (incubating)☆114Updated last year
- RDF-Centric Map/Reduce Framework and Freebase data conversion tool☆149Updated 3 years ago
- Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.☆46Updated 7 years ago
- Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.☆283Updated 7 years ago
- Automatic tagging and analysis of documents in an Apache Solr index for faceted search by RDF(S) Ontologies & SKOS thesauri☆47Updated 3 years ago
- The WikiBrain Java library enables researchers and developers to incorporate state-of-the-art Wikipedia-based algorithms and technologies…☆95Updated 7 years ago
- SKOS Support for Apache Lucene and Solr☆56Updated 4 years ago
- Approve or reject statements from third-party datasets☆146Updated 7 years ago
- Solr Dictionary Annotator (Microservice for Spark)☆71Updated 5 years ago
- The linked open dataset described at http://datahub.io/dataset/vu-wordnet, and the tools used to create it☆25Updated 4 years ago
- An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed…☆152Updated last month
- Entity Extraction Text Processor☆148Updated last year
- ☆185Updated 6 years ago
- Solr AutoComplete implementation☆59Updated 7 years ago
- ImageCat is an Apache OODT RADIX application that uses Apache Solr, Apache Tika and Apache OODT to ingest 10s of millions of files (image…☆96Updated 7 years ago
- Combines Apache OpenNLP and Apache Tika and provides facilities for automatically deriving sentiment from text.☆34Updated 2 years ago
- A queue-controlled browser automation tool for improving web crawl quality☆62Updated last month
- Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in t…☆128Updated 2 months ago
- A text tagger based on Lucene / Solr, using FST technology☆177Updated last year
- Java library for reading and writing WARC files with a typed API☆50Updated last week
- Hadoop jobs for WikiReverse project. Parses Common Crawl data for links to Wikipedia articles.☆38Updated 7 years ago
- XPath extension for extraction from interactive web sites. NOTE: This code is currently out of sync. A more recent, but precompiled versi…☆27Updated 12 years ago
- An RDF plugin for Solr☆115Updated 7 months ago
- General Architecture for Text Engineering☆49Updated 9 years ago
- Examples of Solr configuration entries for Solr plugins and Conceptual Search\Semantic Search from Simon Hughes Dice.com☆26Updated 8 years ago
- An intelligent reading agent that understands text and translates it into Wikidata statements.☆116Updated 9 years ago
- Creates a Neo4j graph of Wikipedia links.☆256Updated 7 years ago
- RDF store on a cloud-based architecture (previously on https://code.google.com/p/cumulusrdf)☆31Updated 9 years ago