Miscellaneous tools for processing WARC files from the CommonCrawl
☆25Jan 1, 2014Updated 12 years ago
Alternatives and similar repositories for warc-tools
Users that are interested in warc-tools are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- warc and wet support for Hadoop's mapreduce api☆14Mar 26, 2015Updated 11 years ago
- All-in-one text tokenizer for Go. Super-fast. Lots of features.☆13Dec 18, 2015Updated 10 years ago
- Read and write WARC files in Go☆50Apr 9, 2018Updated 8 years ago
- Python script to create CDX index files of WARC data☆22Sep 4, 2025Updated last year
- Generate changelog reports based on git log and release tags☆14Jan 16, 2018Updated 8 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- builds the PubChem Bioassay database that is distributed for use with bioassayR☆11Dec 4, 2021Updated 4 years ago
- Common web archive utility code.☆66Updated this week
- Citation Tools☆13May 15, 2017Updated 9 years ago
- ggtikz: Add TikZ annotations with absolute data or relative plot coordinates to ggplots☆11Nov 6, 2025Updated last year
- An example implementatation of synchronized queue for inter-process communication in shared memory☆13Feb 17, 2017Updated 9 years ago
- This is the product of my insomnia.☆11Jul 19, 2012Updated 14 years ago
- plibsys rust bindgen binding☆12Jun 24, 2016Updated 10 years ago
- A simple URL shortener, written in Rust and gRPC☆14Jul 15, 2019Updated 7 years ago
- some scripts and things for processing scanned books☆13Nov 28, 2013Updated 12 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Another Rust BitTorrent implementation☆14Mar 31, 2024Updated 2 years ago
- The Guardian's configuration library☆14Updated this week
- scanning script for the noisebridge book scanner☆14May 12, 2017Updated 9 years ago
- VisNAB☆12Apr 5, 2013Updated 13 years ago
- Go module for accessing Apple Health exported data.☆22Apr 15, 2023Updated 3 years ago
- Python package developed to evaluate textual overlap (N-Grams) between two volumes of text.☆11Sep 23, 2021Updated 5 years ago
- Google Cloud Platform support for Upspin☆13Apr 20, 2024Updated 2 years ago
- Example how to pre-process news articles with textbox and index on Elastic Search☆13Jul 28, 2017Updated 9 years ago
- Martini middleware/handler for parsing Gold templates and rendering HTML☆15May 21, 2014Updated 12 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Example of a serverless web reconaissance workflow's AWS architecture.☆11Feb 25, 2023Updated 3 years ago
- Python bindings for nitro☆16Sep 18, 2013Updated 13 years ago
- Go (golang) simple logger package.☆13Sep 21, 2017Updated 9 years ago
- Experimenting with eager evaluation of Go WASM code☆12Oct 22, 2018Updated 8 years ago
- Easily monitor progress of io operations.☆42Apr 15, 2015Updated 11 years ago
- Promise is a gopherjs-oriented library to help bridge asynchronous Go code and JS via promises.☆11Jan 10, 2020Updated 6 years ago
- Personal notes about different things. Some kind of blog on GitHub.☆14Jun 21, 2021Updated 5 years ago
- Cryptocurrency(Bitcoin Lending Platform) built with Laravel☆11May 16, 2019Updated 7 years ago
- A framework for building your own configuration management tool☆13Oct 6, 2014Updated 12 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Open Journal for Astrophysics (DEPRECATED). Check out this instead: https://github.com/arfon/theoj☆13May 6, 2014Updated 12 years ago
- Documents and wiki pages relevant to the business of the METS Editorial Board☆14Aug 19, 2026Updated 2 months ago
- Pseudo-grammatical Passphrase Generator☆15Jan 26, 2015Updated 11 years ago
- File management for the masses☆13Feb 24, 2015Updated 11 years ago
- Sodoku solver in Go. Based on Peter Norvig's constraint propagation and search algorithm.☆14Mar 14, 2016Updated 10 years ago
- Normalizes casing, spacing & punctuation in a paragraph of text☆14Nov 27, 2015Updated 10 years ago
- CIDR to IP List Tool☆13Mar 12, 2016Updated 10 years ago