CommonCrawl WARC/WET/WAT examples and processing code for Java + Hadoop
☆38Oct 1, 2026Updated last month
Alternatives and similar repositories for cc-warc-examples
Users that are interested in cc-warc-examples are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Launch AWS Elastic MapReduce jobs that process Common Crawl data.☆49Feb 15, 2017Updated 9 years ago
- Common web archive utility code.☆66Oct 1, 2026Updated last month
- DKPro C4CorpusTools is a collection of tools for processing CommonCrawl corpus, including Creative Commons license detection, boilerplate…☆54Jun 12, 2020Updated 6 years ago
- Scientific articles using or citing Common Crawl data☆31Oct 26, 2026Updated last week
- ☆12Jun 10, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Set of scripts to aid in the download of the GDELT data files from www.gdeltproject.org☆12May 17, 2014Updated 12 years ago
- Process Common Crawl data with Python and Spark☆459Sep 17, 2026Updated last month
- Fancy ncurses frontend for opmsg + drops☆11Jul 3, 2017Updated 9 years ago
- Deep Learning, Keras, Tensorflow & Spark Training by Databricks☆10Oct 24, 2017Updated 9 years ago
- Events and Situations Ontology☆15Apr 20, 2018Updated 8 years ago
- A set of reusable Java components that implement functionality common to any web crawler☆259Updated this week
- Hadoop jobs for WikiReverse project. Parses Common Crawl data for links to Wikipedia articles.☆38Aug 12, 2018Updated 8 years ago
- The Dynamic Rules Engine is a serverless application that enables real-time evaluation of rules against sensor data, leveraging AWS Kines…☆11Sep 25, 2024Updated 2 years ago
- A simple Node.js wrapper for the BitX API.☆11Jun 23, 2022Updated 4 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- warc and wet support for Hadoop's mapreduce api☆14Mar 26, 2015Updated 11 years ago
- A library to mutate parquet files☆20May 9, 2023Updated 3 years ago
- ☆16Dec 1, 2021Updated 4 years ago
- Implementation of W3C's R2RML and Direct Mapping specifications☆10Oct 12, 2020Updated 6 years ago
- Virtual patent marking crawler at iproduct.epfl.ch☆15Sep 13, 2017Updated 9 years ago
- Graph benchmarks for Ladybug, Kuzu, lance-csr and DuckPGQ☆14Updated this week
- Natural language detection, Java bindings for CLD2☆20Aug 2, 2026Updated 3 months ago
- Implementation of DSGAN (not fully completed)☆13Dec 28, 2019Updated 6 years ago
- A Hadoop job that runs GATE applications☆15Oct 16, 2013Updated 13 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Declarative syntax for defining sets of URLs. No need for error-prone regexs.☆20Apr 24, 2019Updated 7 years ago
- A bidirectional LSTM example for sequence labeling.☆13May 23, 2018Updated 8 years ago
- JSF 2 tag library containing various components to create fancy diagrams☆13Nov 18, 2013Updated 12 years ago
- FPGA-based LED array driver using HDMI video input☆15Jul 13, 2020Updated 6 years ago
- Word analysis, by domain, on the Common Crawl data set for the purpose of finding industry trends☆57Jan 28, 2024Updated 2 years ago
- Ready-to-use examples of dkpro-core components and pipelines.☆34Dec 16, 2023Updated 2 years ago
- Simple, light and correct LevelDB live read stream implementation☆13Feb 26, 2020Updated 6 years ago
- This repository contains a series of 4 jupyter notebooks demonstrating how AWS AI Services like Amazon Rekognition, Amazon Transcribe and…☆12Nov 26, 2021Updated 4 years ago
- Set of scripts to aid in the download of the GDELT data files from gdelt.utdallas.edu☆18May 14, 2014Updated 12 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Kylo integration with PDND (previously DAF).☆19Nov 16, 2022Updated 3 years ago
- AWS Lambda layer containing latest version of Apache Tika☆14Aug 22, 2026Updated 2 months ago
- ☆15Sep 8, 2016Updated 10 years ago
- A simple CDK app written in Kotlin using Gradle DSL☆12Dec 28, 2018Updated 7 years ago
- Java library for reading and writing WARC files with a typed API☆64Sep 1, 2026Updated 2 months ago
- (formerly-in-use) Annotation-Driven Indexing and Searching with Lucene and Solr☆11Oct 13, 2020Updated 6 years ago
- Two simple cheatsheets about graphs in SPARQL☆15May 29, 2020Updated 6 years ago