playing around with the common crawl dataset
☆70Aug 18, 2012Updated 14 years ago
Alternatives and similar repositories for common-crawl
Users that are interested in common-crawl are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- common crawl quick hack examples☆19Feb 11, 2015Updated 11 years ago
- machine learning experiments around a bunch of rss articles☆14May 10, 2009Updated 17 years ago
- bigram / trigram analysis of wikipedia; mainly mutual info☆22Mar 6, 2012Updated 14 years ago
- trying shingling / resemblance / simhash / sketching to do some data deduping☆96Aug 21, 2015Updated 11 years ago
- do all first links on wikipedia _really_ lead to philosophy?☆23Feb 6, 2013Updated 13 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- A Ruby Temporal Algebra. Describe recurrence with natural language!☆13Apr 6, 2011Updated 15 years ago
- Machine learning and natural language processing with Apache Pig☆53Dec 17, 2013Updated 12 years ago
- latent semantic analysis experiments☆19Apr 20, 2010Updated 16 years ago
- Random projection library for Python, converting a dictionary to low-dimensional numpy matrix☆18Aug 5, 2010Updated 16 years ago
- distributed twitter search engine☆78Jul 27, 2011Updated 15 years ago
- ☆16Jan 15, 2013Updated 13 years ago
- clojure utilities to build training corpora for machine learning / NLP out of public wikimedia dumps: status - partially stalled - will p…☆15Jun 9, 2011Updated 15 years ago
- Swipeable card based layout for Ionic and Angular☆11Jun 30, 2014Updated 12 years ago
- Apache Pig utilities to build training corpora for machine learning / NLP out of public Wikipedia and DBpedia dumps.☆163Nov 8, 2022Updated 4 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Java implementation of a probabilistic set data structure☆145May 18, 2017Updated 9 years ago
- Common Crawl support library to access 2008-2012 crawl archives (ARC files)☆507Nov 29, 2017Updated 8 years ago
- Port FuseFS to Ruby1.9☆12Oct 20, 2011Updated 15 years ago
- Fork of flaxcode htmltotext module☆13Jul 30, 2011Updated 15 years ago
- Bulk loading for elastic search☆185Dec 16, 2023Updated 2 years ago
- CommonCrawl Hello World example☆33Jun 25, 2014Updated 12 years ago
- Examples of use of pig scripting languages capabilities☆39Aug 1, 2016Updated 10 years ago
- Linking Entities in CommonCrawl Dataset onto Wikipedia Concepts☆59Sep 5, 2012Updated 14 years ago
- Preprocess text for NLP (tokenizing, lowercasing, stemming, sentence splitting, etc.)☆29Jun 7, 2011Updated 15 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- A reporistory of User-defined functions for Apache Pig☆16Sep 20, 2010Updated 16 years ago
- Behemoth is an open source platform for large scale document analysis based on Apache Hadoop.☆282Apr 25, 2018Updated 8 years ago
- Slinky, a high-performance web crawler / text analytics in Python, Redis, Hadoop, R, Gephi☆40Aug 30, 2010Updated 16 years ago
- Classias: a collection of machine-learning algorithms for classification☆14Jul 24, 2019Updated 7 years ago
- Chain stream operations together☆15Oct 21, 2012Updated 14 years ago
- State-of-The-Art Unsupervised Part-Of-Speech Type-Level Tagger in 300 Lines of Clojure☆41Sep 15, 2010Updated 16 years ago
- Basic implementation of a gossip protocol. This is a porting of Java implementation. see http://code.google.com/p/gossip-protocol-java/☆13Sep 7, 2013Updated 13 years ago
- The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)☆226Dec 22, 2022Updated 3 years ago
- *Deprecated* A short and sweet Python web crawler using Redis as the process queue, seen set and Memcache style rate limiter for robots.t…☆16Feb 22, 2011Updated 15 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- Yahoo!'s topic modelling framework using Latent Dirichlet Allocation☆337Sep 21, 2011Updated 15 years ago
- A platform for storing large semantic networks on MongoDB☆21Jun 20, 2011Updated 15 years ago
- A Clojure client for Apache Solr☆31Sep 25, 2013Updated 13 years ago
- A neat and flexible pelican theme☆12Aug 8, 2014Updated 12 years ago
- A Text Atlas!☆29Nov 22, 2010Updated 15 years ago
- Experiments for the publication 'Scalable Similarity-Based Neighborhood Methods with MapReduce' at the ACM Recommender Systems Conference…☆14Jun 6, 2012Updated 14 years ago
- ruby bayes naive classifier implementation that uses Redis as backend☆22May 26, 2010Updated 16 years ago
Are these results useful?