rossf7 / elasticrawlLinks
Launch AWS Elastic MapReduce jobs that process Common Crawl data.
☆49Updated 8 years ago
Alternatives and similar repositories for elasticrawl
Users that are interested in elasticrawl are comparing it to the libraries listed below
Sorting:
- Human-Powered Data Analysis with Mechanical Turk☆300Updated 12 years ago
- Index URLs in Common Crawl☆194Updated 7 years ago
- Demonstration of using Python to process the Common Crawl dataset with the mrjob framework☆167Updated 3 years ago
- Keeps a mirror of DBpedia live in sync☆26Updated 3 years ago
- A command-line tool for using CommonCrawl Index API at http://index.commoncrawl.org/☆196Updated 6 years ago
- RDF-Centric Map/Reduce Framework and Freebase data conversion tool☆149Updated 3 years ago
- Wikipedia Live Monitor☆22Updated 8 months ago
- Hadoop jobs for WikiReverse project. Parses Common Crawl data for links to Wikipedia articles.☆38Updated 7 years ago
- Text classification using Naive Bayes and Elasticsearch☆154Updated 9 years ago
- Creates a Neo4j graph of Wikipedia links.☆256Updated 7 years ago
- Extract postal addresses from the DOM☆66Updated 13 years ago
- The Summarizer from the Web IR / NLP Group (WING), hence SWING, is a modular, state-of-the-art automatic extractive text summarization sy…☆39Updated 10 years ago
- Model Training tool for MITIE☆79Updated 10 years ago
- ☆185Updated 6 years ago
- trying shingling / resemblance / simhash / sketching to do some data deduping☆98Updated 10 years ago
- KEA 5.0 (keyphrase extraction software), modified to be an XML-RPC service☆42Updated 14 years ago
- Evaluating the performance and accuracy of ABBYY FineReader's OCR on Senate Financial Disclosure scanned forms☆133Updated 9 years ago
- Elasticsearch/Solr Sandbox for exploring explain information and tweaking☆137Updated last year
- Wikidata and Wikipedia API client.☆35Updated last year
- ☆29Updated 8 years ago
- Additional opennlp mapping type for elasticsearch in order to perform named entity recognition☆136Updated 9 years ago
- Traptor -- A distributed Twitter feed☆26Updated 2 years ago
- Solrstrap is a Query-Result interface for Solr written in JavaScript, HTML and CSS☆87Updated 8 years ago
- Semanticizest: dump parser and client☆20Updated 9 years ago
- FacetView is a pure javascript frontend for ElasticSearch.☆291Updated 10 years ago
- Open source large document set visualization platform☆270Updated 2 years ago
- Algorithmic summarizer for RSS/Atom Feeds, Web Urls and arbitrary text. Codebase for the application deployed at http://tldrzr.herokuapp.…☆53Updated 8 years ago
- Lentil is no longer supported. Lentil is a Ruby on Rails Engine that supports the harvesting of images from Instagram.☆58Updated 6 years ago
- Pipeline for distributed Natural Language Processing, made in Python☆65Updated 8 years ago
- Supervised learning for novelty detection in text☆78Updated 8 years ago