Python script to create CDX index files of WARC data
☆22Sep 4, 2025Updated last year
Alternatives and similar repositories for CDX-Writer
Users that are interested in CDX-Writer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Python library for reading and writing warc files☆251Mar 7, 2022Updated 4 years ago
- ☆23Feb 22, 2024Updated 2 years ago
- Centralised repository for WARC usage specifications.☆139Sep 20, 2026Updated last month
- Sort-friendly URI Reordering Transform (SURT) python module☆48Sep 11, 2025Updated last year
- Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)☆178Aug 18, 2025Updated last year
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- The Seesaw pipeline grab script for the URLTeam (terroroftinytown) project☆28Jul 17, 2025Updated last year
- ☆59Jan 6, 2023Updated 3 years ago
- The UKWA Heritrix3 custom modules and Docker builder.☆10Dec 2, 2024Updated last year
- Java library implementing the IIIF Presentation API☆15Feb 28, 2018Updated 8 years ago
- Wget with Lua extension☆24Dec 17, 2015Updated 10 years ago
- Object Resource Stream and CDXJ Drafts☆15Nov 28, 2018Updated 7 years ago
- Go package providing tools for working with Library of Congress data.☆12Oct 3, 2024Updated 2 years ago
- Convert Directories, Files and ZIP Files to Web Archives (WARC)☆102Aug 28, 2026Updated 2 months ago
- Web archiving using Google Chrome☆44Dec 30, 2019Updated 6 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- WARC writing MITM HTTP/S proxy☆468Jun 17, 2026Updated 4 months ago
- Prototype SOLR-powered web archive exploration UI.☆44Jun 3, 2020Updated 6 years ago
- Demo app built using AngularJS with Backand serving as the back end☆13Mar 1, 2017Updated 9 years ago
- Backend, IA-specific tools for crawling and processing the scholarly web. Content ends up in https://fatcat.wiki☆28Jul 31, 2024Updated 2 years ago
- The OpenWayback Development☆530Jan 3, 2024Updated 2 years ago
- Puzzle language written in Linear A☆12Mar 29, 2018Updated 8 years ago
- A tool for detecting viruses and NSFW material in WARC files☆18Jul 15, 2026Updated 3 months ago
- Archive Research Services Workshop☆31Sep 29, 2017Updated 9 years ago
- "The Pilotbird (Pycnoptilus floccosus) acts like nothing is wrong, but maybe she's just really good at lying."☆11Oct 23, 2026Updated last week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- CDXJ Indexing of WARC/ARCs☆37Updated this week
- WASAPI data transfer APIs☆51Apr 23, 2022Updated 4 years ago
- Chrome extension that uses Memento to indicate that a page a user is viewing on the live web has an archived copy and to give the user ac…☆59Aug 27, 2025Updated last year
- The Guardian's configuration library☆12Updated this week
- Backend scripts, files, etc. for parsing/updating dictionaries.☆19Jun 22, 2015Updated 11 years ago
- IA's public Wayback Machine (moved from SourceForge)☆868Mar 1, 2024Updated 2 years ago
- Apache Annotator (Incubating) Website☆11Aug 11, 2025Updated last year
- 信息安全方面rss订阅☆12May 29, 2022Updated 4 years ago
- A REST webservice to access the ledger cli accounting system☆11Jan 1, 2014Updated 12 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Tools to analyze web archives☆20Jul 12, 2016Updated 10 years ago
- code and data used to build a training dataset for dragnet models☆10Nov 29, 2020Updated 5 years ago
- Python package developed to evaluate textual overlap (N-Grams) between two volumes of text.☆11Sep 23, 2021Updated 5 years ago
- An open-source pipeline developed by the Library of Congress for reprocessing the OCR of National Digital Newspaper Program (NDNP) data. …☆32Oct 24, 2026Updated last week
- family finance tools of a closet librarian☆13Feb 7, 2026Updated 8 months ago
- An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed…☆164Oct 8, 2025Updated last year
- TypeDB (Core and Cloud) RPC Communication Protocol☆19Updated this week