π A compilation of research relevant to Data Together's efforts tackling the general problem of data resilience & interactivity
β100Sep 27, 2018Updated 8 years ago
Alternatives and similar repositories for research
Users that are interested in research are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Parallelized web crawler written in Golangβ15Oct 2, 2018Updated 8 years ago
- Web application to allow users to add content metadata about crawled resourcesβ13Feb 15, 2018Updated 8 years ago
- Convert Directories, Files and ZIP Files to Web Archives (WARC)β103Aug 28, 2026Updated 2 months ago
- β10Jun 10, 2016Updated 10 years ago
- Hacking challenges to learn web archive security.β35Jun 23, 2017Updated 9 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer β’ AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- An Awesome List for getting started with web archivingβ2,692Nov 6, 2026Updated 2 weeks ago
- Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a headβ178May 19, 2020Updated 6 years ago
- A tool to scrape the ipfs network for information on the number of peers in the network.β21Mar 22, 2024Updated 2 years ago
- utility to fetch provenance information from Internet Archive's Wayback Machineβ15Feb 5, 2026Updated 9 months ago
- Official Python package for ArchiveBox, the self-hosted internet archiving solution.β12Oct 5, 2024Updated 2 years ago
- Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in tβ¦β134Nov 21, 2025Updated 11 months ago
- Tools for access, "diff"-ing, and analyzing archived web pagesβ24Updated this week
- brozzler - distributed browser-based web crawlerβ824Updated this week
- A dockerized, queued high fidelity web archiver based on Squidwarcβ62Jul 9, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer β’ AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Visegrad+ Parliament API. Access to parliament data of Visegrad+ countries in a common data standard.β11May 17, 2016Updated 10 years ago
- linkbak is a web page archiver : it reads a list of links and dumps the corresponding pages in HTML and PDF.β13Dec 8, 2022Updated 3 years ago
- A list of things related to software, literature, and other content for π£ Mementoβ126May 22, 2026Updated 5 months ago
- A ServiceWorker for client-side reconstruction of composite mementosβ16Mar 6, 2025Updated last year
- Collaborative collection development for web archivesβ19Sep 5, 2019Updated 7 years ago
- Host, backup, and share hyperdrive archivesβ13Aug 22, 2017Updated 9 years ago
- URL-agnostic WARC dedupe serverβ12Jun 6, 2026Updated 5 months ago
- A service that provides archive-aware oEmbed-compatible embeddable surrogates (social cards, thumbnails, etc.) for archived web pages (meβ¦β14Nov 15, 2021Updated 5 years ago
- Tools for helping you work with web platform archive downloads.β18Mar 27, 2020Updated 6 years ago
- Managed Kubernetes at scale on DigitalOcean β’ AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Start here! Discussion for Data Together: Building a better future for dataβ47Oct 21, 2019Updated 7 years ago
- simple dat downloading moduleβ10Jan 11, 2023Updated 3 years ago
- Request For Comments (RFCs) documenting changes to Qriβ12Nov 23, 2021Updated 4 years ago
- An HTTP API for tracking and annotating changes to a set of web pages.β24Updated this week
- Clone the NPM registry into IPFSβ20May 1, 2025Updated last year
- Web archiving using Google Chromeβ44Dec 30, 2019Updated 6 years ago
- Webrecorder Player for Desktop (OSX/Windows/Linux). (Built with Electron + Webrecorder)β445Sep 17, 2020Updated 6 years ago
- Web Archiving Integration Layer: One-Click User Instigated Preservationβ404Jun 19, 2026Updated 5 months ago
- Implementation of the Block data structure in JavaScript.β13Dec 6, 2022Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI β’ AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- An easy-to-use and highly customizable crawler that enables you to create your own little Web archives (WARC/CDX)β27Oct 9, 2017Updated 9 years ago
- WARC writing MITM HTTP/S proxyβ470Jun 17, 2026Updated 5 months ago
- Parse OCR result files for pagenos, tables of contents, etc.β14Nov 30, 2011Updated 14 years ago
- A Memento Aggregator CLI and Server in Goβ83Apr 9, 2026Updated 7 months ago
- Uploads items into the Internet Archive after they have been downloaded with youtube-dlβ15Nov 11, 2026Updated last week
- Merging WARCs into a single WARC fileβ15Aug 29, 2014Updated 12 years ago
- Dat link resolver [ DEPRECATED - More info on active projects and modules at https://dat-ecosystem.org/ ]β18Jan 6, 2022Updated 4 years ago
Are these results useful?