Alternatives and similar repositories for CommonCrawlDocumentDownload:

Users that are interested in CommonCrawlDocumentDownload are comparing it to the libraries listed below

• tballison / commoncrawl-fetcher-lite
  Simplified version of a common crawl fetcher
  ☆13Updated this week
• dkpro / dkpro-c4corpus
  DKPro C4CorpusTools is a collection of tools for processing CommonCrawl corpus, including Creative Commons license detection, boilerplate…
  ☆50Updated 4 years ago
• mitre / rhapsode
  Advanced desktop search/corpus exploration prototype
  ☆21Updated 3 years ago
• thammegowda / tika-ner-corenlp
  Stanford CoreNLP NER addon for Apache Tika's NamerEntityParser
  ☆13Updated 2 years ago
• ikreymer / cc-index-server
  Deployment of pywb as a CommonCrawl Index Server
  ☆21Updated 7 years ago
• ikreymer / webarchive-indexing
  Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
  ☆43Updated 7 years ago
• alephdata / synonames
  Trying to generate name synonyms from wikidata
  ☆32Updated 4 years ago
• alephdata / ingest-file
  Ingestors extract the contents of mixed unstructured documents into structured (followthemoney) data.
  ☆58Updated 2 weeks ago
• ckorzen / pdf-text-extraction-benchmark
  A project about benchmarking and evaluating existing PDF extraction tools on their semantic abilities to extract the body texts from PDF …
  ☆66Updated 4 years ago
• zelandiya / maui-standalone
  ☆21Updated 6 years ago
• openpreserve / format-corpus
  An openly-licensed corpus of small example files, covering a wide range of formats and creation tools.
  ☆188Updated last year
• usc-isi-i2 / etk
  Extraction Toolkit
  ☆82Updated 3 years ago
• commoncrawl / cc-index-server
  Common Crawl Index Server
  ☆65Updated this week
• kermitt2 / grobid-ner
  A Named-Entity Recogniser based on Grobid.
  ☆50Updated 4 months ago
• lfoppiano / grobid-quantities
  GROBID extension for identifying and normalizing physical quantities.
  ☆77Updated 4 months ago
• dbmdz / solr-ocrpayload-plugin
  Efficient indexing and retrieval of OCR bounding boxes in Solr
  ☆22Updated 5 years ago
• iipc / webarchive-commons
  Common web archive utility code.
  ☆52Updated last month
• trivio / common_crawl_index
  Index URLs in Common Crawl
  ☆194Updated 7 years ago
• U-Alberta / exemplar
  An open relation extraction system
  ☆46Updated 3 years ago
• LanguageMachines / PICCL
  A set of workflows for corpus building through OCR, post-correction and normalisation
  ☆48Updated 2 years ago
• chrismattmann / tika-similarity
  Tika-Similarity uses the Tika-Python package (Python port of Apache Tika) to compute file similarity based on Metadata features.
  ☆108Updated 10 months ago
• TeamHG-Memex / sitehound
  This is the facade for installation and access to the individual components
  ☆16Updated 6 years ago
• Sotera / pst-extraction
  PST extraction and analytic pipeline
  ☆37Updated 6 years ago
• USCDataScience / autoextractor
  A toolkit for clustering web pages based on various similarity measures.
  ☆33Updated 3 years ago
• DiceTechJobs / VectorsInSearch
  Dice.com repo to accompany the dice.com 'Vectors in Search' talk by Simon Hughes, from the Activate 2018 search conference, and the 'Sear…
  ☆85Updated 3 years ago
• marcusklang / wikiforia
  A Utility Library for Wikipedia dumps
  ☆33Updated 7 years ago
• fedelemantuano / tika-app-python
  Python bindings for Apache Tika
  ☆22Updated 4 years ago
• cmharlow / GetUrRecon
  All that entity matching, resolution, normalization, enhancement and reconciliation madness, but with a focus on data, not platforms.
  ☆24Updated 2 years ago
• sematext / query-segmenter
  Solr Query Segmenter for structuring unstructured queries
  ☆21Updated 3 years ago
• VIDA-NYU / domain_discovery_tool
  This repository contains the Domain Discovery Tool (DDT) project. DDT is an interactive system that helps users explore and better unders…
  ☆46Updated 3 years ago