Alternatives and similar repositories for cc-index-server

Users that are interested in cc-index-server are comparing it to the libraries listed below

• cocrawler / cdx_toolkit
  A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine
  ☆183Updated 7 months ago
• VIDA-NYU / domain_discovery_tool
  This repository contains the Domain Discovery Tool (DDT) project. DDT is an interactive system that helps users explore and better unders…
  ☆46Updated 3 years ago
• ikreymer / cc-index-server
  Deployment of pywb as a CommonCrawl Index Server
  ☆21Updated 7 years ago
• darpa-i2o / memex-program-index
  A list of memex-related tools and their repository URLs
  ☆152Updated 7 years ago
• ikreymer / cdx-index-client
  A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/
  ☆196Updated 6 years ago
• commoncrawl / cc-crawl-statistics
  Statistics of Common Crawl monthly archives mined from URL index files
  ☆188Updated 2 weeks ago
• openpreserve / pagelyzer
  Suite of tools for detecting changes in web pages and their rendering
  ☆55Updated last year
• commoncrawl / cc-webgraph
  Tools to construct and process Common Crawl webgraphs
  ☆93Updated last week
• CI-Research / KeywordAnalysis
  Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends
  ☆57Updated last year
• anvaka / yasiv-youtube
  Graph of related videos from YouTube
  ☆110Updated 2 years ago
• opensemanticsearch / solr-relevance-ranking-analysis
  Solr Relevance Ranking Analysis and Visualization Tool
  ☆15Updated 5 years ago
• rflynn / regroup
  Generate a regular expression that describes a set of strings.
  ☆30Updated 2 years ago
• thammegowda / tika-ner-corenlp
  Stanford CoreNLP NER addon for Apache Tika's NamerEntityParser
  ☆13Updated 3 years ago
• iproduct-database / vpm-filter-spark
  Virtual patent marking crawler at iproduct.epfl.ch
  ☆15Updated 7 years ago
• commoncrawl / cc-index-table
  Index Common Crawl archives in tabular format
  ☆122Updated last month
• commoncrawl / cc-warc-examples
  CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop
  ☆37Updated 8 months ago
• mattbierner / urban-dictionary-word-list
  Script and sample dataset of all urban dictionary entry names (around 1.4 million total)
  ☆91Updated 3 years ago
• USCDataScience / SentimentAnalysisParser
  Combines Apache OpenNLP and Apache Tika and provides facilities for automatically deriving sentiment from text.
  ☆34Updated 2 years ago
• medialab / hyphe-browser
  Browser version of Hyphe (WIP)
  ☆31Updated 3 months ago
• VIDA-NYU / ache
  ACHE is a web crawler for domain-specific search.
  ☆471Updated last week
• anvaka / allgithub
  Crawling github data
  ☆31Updated last year
• Lab41 / altair
  Assessing Source Code Semantic Similarity with Unsupervised Learning
  ☆41Updated 7 years ago
• dkpro / dkpro-c4corpus
  DKPro C4CorpusTools is a collection of tools for processing CommonCrawl corpus, including Creative Commons license detection, boilerplate…
  ☆52Updated 5 years ago
• opensemanticsearch / open-semantic-visual-graph-explorer
  Open Semantic Visual Linked Data Graph Explorer: Open Source tool (web app) and user interace (UI) for discovery, exploration and visuali…
  ☆85Updated 5 years ago
• sangaline / scrapy-wayback-machine
  A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
  ☆116Updated last year
• TeamHG-Memex / sitehound
  This is the facade for installation and access to the individual components
  ☆15Updated 7 years ago
• mitll / MITIE
  MITIE: library and tools for information extraction
  ☆29Updated 10 years ago
• usc-isi-i2 / etk
  Extraction Toolkit
  ☆83Updated 3 years ago
• GateNLP / gate-core
  The GATE Embedded core API and GATE Developer application
  ☆88Updated 9 months ago
• ikreymer / webarchive-indexing
  Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
  ☆46Updated 7 years ago