Python script to create CDX index files of WARC data
☆22Sep 4, 2025Updated 11 months ago
Alternatives and similar repositories for CDX-Writer
Users that are interested in CDX-Writer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- A JavaScript port of most of Rainbow (conanite's JVM-based Arc implementation).☆22Feb 1, 2022Updated 4 years ago
- Python library for reading and writing warc files☆249Mar 7, 2022Updated 4 years ago
- The Seesaw pipeline grab script for the URLTeam (terroroftinytown) project☆28Jul 17, 2025Updated last year
- Backend, IA-specific tools for crawling and processing the scholarly web. Content ends up in https://fatcat.wiki☆28Jul 31, 2024Updated 2 years ago
- A tool for detecting viruses and NSFW material in WARC files☆18Jul 15, 2026Updated last month
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Demo app built using AngularJS with Backand serving as the back end☆13Mar 1, 2017Updated 9 years ago
- ☆15Sep 8, 2016Updated 9 years ago
- Tools to analyze web archives☆20Jul 12, 2016Updated 10 years ago
- An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed…☆162Oct 8, 2025Updated 10 months ago
- GitHub's month-long game jam November 2016☆10May 11, 2026Updated 3 months ago
- Specification for a query language to request Verifiable Presentations from wallets etc.☆10Apr 23, 2026Updated 4 months ago
- code and data used to build a training dataset for dragnet models☆10Nov 29, 2020Updated 5 years ago
- An Image Dictionary for Co-dfns☆14Jun 16, 2017Updated 9 years ago
- arc in java☆59Oct 12, 2010Updated 15 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- (Note: This repository is obsolete, please see the new Browsertrix webrecorder/browsertrix) Browser-Based On-Demand Web Archiving Automat…☆38Apr 23, 2019Updated 7 years ago
- An IETF specification for cryptographic hyperlinking☆15May 2, 2021Updated 5 years ago
- You've made the list, we'll help you check it twice. Given a domain-like string, verifies inclusion in a list you provide.☆19Nov 13, 2020Updated 5 years ago
- External link tracking tool for Wikimedia partnerships☆11Updated this week
- CDXJ Indexing of WARC/ARCs☆36Aug 19, 2026Updated last week
- ARK minter, binder, resolver☆24May 28, 2026Updated 3 months ago
- TypeDB (Core and Cloud) RPC Communication Protocol☆18Aug 7, 2026Updated 3 weeks ago
- Linked Data Signatures for JWS☆13Aug 5, 2022Updated 4 years ago
- A service that provides archive-aware oEmbed-compatible embeddable surrogates (social cards, thumbnails, etc.) for archived web pages (me…☆14Nov 15, 2021Updated 4 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- A queue-controlled browser automation tool for improving web crawl quality☆68May 28, 2026Updated 3 months ago
- A complete workflow for converting API Blueprint to hosted documentation on S3☆15Jan 15, 2020Updated 6 years ago
- Jupyter Notebooks Relating to Open Context (https://opencontext.org)☆11Aug 7, 2026Updated 3 weeks ago
- Kaitai Struct YAML (KSY) schema specification☆17Sep 12, 2025Updated 11 months ago
- little scripts to introduce people to each other.☆20Jun 25, 2016Updated 10 years ago
- Builders for attrs☆11Jul 31, 2019Updated 7 years ago
- Developer Documentation for dev.mixer.com☆12Jun 11, 2020Updated 6 years ago
- GUI toolkit for Ada using OpenGL☆15Jan 17, 2013Updated 13 years ago
- Networking library based on anyio☆10Jun 29, 2026Updated 2 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Docs for the public Partnerize API☆12May 7, 2021Updated 5 years ago
- KSFL - Kreative Structured Format Library☆17Feb 21, 2023Updated 3 years ago
- This tool will create JavaScript facades for Scala.js from typescript ambient declarations.☆18Feb 4, 2018Updated 8 years ago
- This Chrome extension save all web pages you viewed to the Wayback Machine☆14Mar 7, 2021Updated 5 years ago
- WARC writing MITM HTTP/S proxy☆464Jun 17, 2026Updated 2 months ago
- Embed Carbon screenshots in your markdown files☆16Jun 29, 2023Updated 3 years ago
- ☆12May 20, 2025Updated last year