playing around with the common crawl dataset
☆70Aug 18, 2012Updated 14 years ago
Alternatives and similar repositories for common-crawl
Users that are interested in common-crawl are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- common crawl quick hack examples☆19Feb 11, 2015Updated 11 years ago
- bigram / trigram analysis of wikipedia; mainly mutual info☆22Mar 6, 2012Updated 14 years ago
- trying shingling / resemblance / simhash / sketching to do some data deduping☆97Aug 21, 2015Updated 11 years ago
- Fork of flaxcode htmltotext module☆13Jul 30, 2011Updated 15 years ago
- A library of examples showing how to use the Common Crawl corpus (2008-2012, ARC format)☆67Aug 5, 2016Updated 10 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Machine learning and natural language processing with Apache Pig☆53Dec 17, 2013Updated 12 years ago
- latent semantic analysis experiments☆19Apr 20, 2010Updated 16 years ago
- Random projection library for Python, converting a dictionary to low-dimensional numpy matrix☆18Aug 5, 2010Updated 16 years ago
- distributed twitter search engine☆78Jul 27, 2011Updated 15 years ago
- Miscellaneous tools in bash, Python, and Perl for munging Big Data.☆17Jun 10, 2010Updated 16 years ago
- Apache Pig utilities to build training corpora for machine learning / NLP out of public Wikipedia and DBpedia dumps.☆163Nov 8, 2022Updated 3 years ago
- clojure utilities to build training corpora for machine learning / NLP out of public wikimedia dumps: status - partially stalled - will p…☆15Jun 9, 2011Updated 15 years ago
- Common Crawl support library to access 2008-2012 crawl archives (ARC files)☆507Nov 29, 2017Updated 8 years ago
- A set of examples and utilities for using Pig with Cassandra. For the latest jar release, check the Downloads link.☆84Aug 21, 2014Updated 12 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Bulk loading for elastic search☆186Dec 16, 2023Updated 2 years ago
- A Hadoop toolkit for web-scale information retrieval research☆86Dec 12, 2014Updated 11 years ago
- CommonCrawl Hello World example☆33Jun 25, 2014Updated 12 years ago
- A reporistory of User-defined functions for Apache Pig☆16Sep 20, 2010Updated 16 years ago
- Linking Entities in CommonCrawl Dataset onto Wikipedia Concepts☆59Sep 5, 2012Updated 14 years ago
- Examples of use of pig scripting languages capabilities☆39Aug 1, 2016Updated 10 years ago
- Makes a submission to Kaggle's wordpress competetition using LDA features☆16Jul 13, 2012Updated 14 years ago
- A common set of compute primitives for PyCUDA and PyOpenCL☆59Jul 16, 2026Updated 2 months ago
- Sample Ruby on Rails application that uses the LinkedIn API along with the Authlogic OAuth extension to create browseable resumes.☆27Jun 3, 2010Updated 16 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- State-of-The-Art Unsupervised Part-Of-Speech Type-Level Tagger in 300 Lines of Clojure☆41Sep 15, 2010Updated 16 years ago
- Dropbox River for Elasticsearch (PROJECT STOPPED)☆20Dec 1, 2015Updated 10 years ago
- Stream-based InputFormat for processing the compressed XML dumps of Wikipedia with Hadoop☆85Jun 8, 2013Updated 13 years ago
- A project to gather, analyze and visualized the data in Crunchbase☆46Nov 8, 2011Updated 14 years ago
- The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)☆226Dec 22, 2022Updated 3 years ago
- A Clojure client for Apache Solr☆31Sep 25, 2013Updated 13 years ago
- A constant-memory approximation of set membership☆25Jan 25, 2025Updated last year
- ruby bayes naive classifier implementation that uses Redis as backend☆22May 26, 2010Updated 16 years ago
- A Text Atlas!☆29Nov 22, 2010Updated 15 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Semantic Web Service Composition Engine☆15Sep 15, 2015Updated 11 years ago
- Shared Memory Numpy ( Deprecated, See https://github.com/ContinuumIO/blaze )☆22Oct 27, 2012Updated 13 years ago
- Collect, store and analyze your Internet provider's usage stats.☆16Sep 27, 2015Updated 11 years ago
- Haskell Powered Doorbell☆12Jul 14, 2019Updated 7 years ago
- S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop ap…☆232Mar 4, 2011Updated 15 years ago
- A catalog of interesting clojure.spec repos and gists☆10Apr 4, 2017Updated 9 years ago
- Roku Channel for Development Videos☆21Aug 15, 2013Updated 13 years ago