rossf7 / elasticrawlLinks
Launch AWS Elastic MapReduce jobs that process Common Crawl data.
☆49Updated 8 years ago
Alternatives and similar repositories for elasticrawl
Users that are interested in elasticrawl are comparing it to the libraries listed below
Sorting:
- Human-Powered Data Analysis with Mechanical Turk☆300Updated 12 years ago
- Index URLs in Common Crawl☆195Updated 8 years ago
- The Summarizer from the Web IR / NLP Group (WING), hence SWING, is a modular, state-of-the-art automatic extractive text summarization sy…☆39Updated 10 years ago
- Evaluating the performance and accuracy of ABBYY FineReader's OCR on Senate Financial Disclosure scanned forms☆134Updated 9 years ago
- Extract postal addresses from the DOM☆66Updated 13 years ago
- Analyze the structure and dynamics of an open source project's developer community, using graph algorithms, etc.☆58Updated 4 years ago
- RDF-Centric Map/Reduce Framework and Freebase data conversion tool☆148Updated 3 years ago
- Model Training tool for MITIE☆79Updated 10 years ago
- ImageCat is an Apache OODT RADIX application that uses Apache Solr, Apache Tika and Apache OODT to ingest 10s of millions of files (image…☆96Updated 7 years ago
- Demonstration of using Python to process the Common Crawl dataset with the mrjob framework☆166Updated 3 years ago
- Traptor -- A distributed Twitter feed☆26Updated 3 years ago
- FacetView is a pure javascript frontend for ElasticSearch.☆291Updated 10 years ago
- trying shingling / resemblance / simhash / sketching to do some data deduping☆97Updated 10 years ago
- A Seriously Fun guide to Big Data Analytics in Practice☆169Updated 10 years ago
- ☆29Updated 8 years ago
- A collection and conversion of WARN notices from California☆12Updated 9 years ago
- Hadoop jobs for WikiReverse project. Parses Common Crawl data for links to Wikipedia articles.☆38Updated 7 years ago
- ☆185Updated 6 years ago
- Parser and standardizer for politician, individual and organization names.☆129Updated 8 years ago
- Dump the Twitter stream to JSON and CSV, then apply filters, reject non-English content, do sentiment analysis, and more.☆66Updated 11 years ago
- A simple Python library/tool for pulling location information from unstructured text☆186Updated 14 years ago
- Updates to Zope's keyphrase extractor (forked from 1.1.0)☆67Updated 8 years ago
- Wikidata and Wikipedia API client.☆35Updated 2 years ago
- Code to transform Hillary's emails from raw PDF documents to a SQLite database☆161Updated 9 years ago
- Akiva is a simple natural-language-processing, question-answering, artificial intelligence.☆347Updated 11 years ago
- Demo visualization of Neo4j data☆162Updated 8 years ago
- A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/☆203Updated 7 years ago
- We introduce TACIT: An Open-Source Text Analysis, Crawling and Interpretation Tool. TACIT's plugin architecture has three main components…☆109Updated 6 years ago
- Mechanical Turk on your own machine.☆207Updated 11 months ago
- Vocabulary using n-grams☆16Updated 7 years ago