Alternatives and similar repositories for elasticrawl

Users that are interested in elasticrawl are comparing it to the libraries listed below

• twitter-archive / clockworkraven
  Human-Powered Data Analysis with Mechanical Turk
  ☆300Updated 12 years ago
• trivio / common_crawl_index
  Index URLs in Common Crawl
  ☆195Updated 7 years ago
• WING-NUS / SWING
  The Summarizer from the Web IR / NLP Group (WING), hence SWING, is a modular, state-of-the-art automatic extractive text summarization sy…
  ☆39Updated 10 years ago
• infochimps-labs / big_data_for_chimps
  A Seriously Fun guide to Big Data Analytics in Practice
  ☆169Updated 10 years ago
• rossf7 / wikireverse
  Hadoop jobs for WikiReverse project. Parses Common Crawl data for links to Wikipedia articles.
  ☆38Updated 7 years ago
• matpalm / resemblance
  trying shingling / resemblance / simhash / sketching to do some data deduping
  ☆98Updated 10 years ago
• commoncrawl / cc-mrjob
  Demonstration of using Python to process the Common Crawl dataset with the mrjob framework
  ☆167Updated 3 years ago
• NCSU-Libraries / lentil
  Lentil is no longer supported. Lentil is a Ruby on Rails Engine that supports the harvesting of images from Instagram.
  ☆58Updated 6 years ago
• tomayac / wikipedia-live-monitor
  Wikipedia Live Monitor
  ☆22Updated 8 months ago
• datahoarder / ca-warn
  A collection and conversion of WARN notices from California
  ☆12Updated 9 years ago
• cantino / twitter_to_csv
  Dump the Twitter stream to JSON and CSV, then apply filters, reject non-English content, do sentiment analysis, and more.
  ☆66Updated 11 years ago
• Sotera / mitie-trainer
  Model Training tool for MITIE
  ☆79Updated 10 years ago
• ikreymer / cdx-index-client
  A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/
  ☆199Updated 6 years ago
• josephwilk / rsemantic
  A document vector search with flexible matrix transforms. Currently supports Latent semantic analysis and Term frequency - inverse docume…
  ☆150Updated 5 years ago
• fergiemcdowall / solrstrap
  Solrstrap is a Query-Result interface for Solr written in JavaScript, HTML and CSS
  ☆87Updated 8 years ago
• lumoslabs / aleph
  Empower Curiosity / Redshift analytics platform
  ☆77Updated 4 years ago
• klacointe / wikidata-client
  Wikidata and Wikipedia API client.
  ☆35Updated last year
• max-mapper / refine-ruby
  Ruby client library for controlling Google Refine
  ☆44Updated 7 years ago
• paulhoule / infovore
  RDF-Centric Map/Reduce Framework and Freebase data conversion tool
  ☆149Updated 3 years ago
• zelandiya / maui
  ☆185Updated 6 years ago
• ikreymer / webarchive-indexing
  Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
  ☆46Updated 7 years ago
• ankane / quirk
  Build powerful predictive models with a few lines of code
  ☆23Updated 2 years ago
• glean / glean
  A data management tool for humans
  ☆119Updated 8 years ago
• rysmith / hashrobot
  A social media assistant.
  ☆49Updated 7 years ago
• documentcloud / pdfshaver
  Shave pages off of PDFs as images
  ☆59Updated 7 years ago
• snowplow-archive / sluice
  A Ruby toolkit for cloud-friendly ETL
  ☆38Updated 9 years ago
• danvk / localturk
  Mechanical Turk on your own machine.
  ☆207Updated 10 months ago
• dbpedia-spotlight / wikipedia-stats-extractor
  Raw Wikipedia counts for entity linking
  ☆19Updated 8 years ago
• chrismattmann / imagecat
  ImageCat is an Apache OODT RADIX application that uses Apache Solr, Apache Tika and Apache OODT to ingest 10s of millions of files (image…
  ☆96Updated 7 years ago
• turian / kea-service
  KEA 5.0 (keyphrase extraction software), modified to be an XML-RPC service
  ☆42Updated 14 years ago