internetarchive / warcprox
WARC writing MITM HTTP/S proxy
☆401Updated 2 weeks ago
Alternatives and similar repositories for warcprox:
Users that are interested in warcprox are comparing it to the libraries listed below
- Chrome extension to "Create WARC files from any webpage"☆220Updated last year
- brozzler - distributed browser-based web crawler☆702Updated this week
- Web Archiving Integration Layer: One-Click User Instigated Preservation☆372Updated last month
- Wget-compatible web downloader and crawler.☆583Updated 11 months ago
- The OpenWayback Development☆497Updated last year
- Core Python Web Archiving Toolkit for replay and recording of web archives☆1,489Updated this week
- Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head☆170Updated 4 years ago
- Webrecorder Player for Desktop (OSX/Windows/Linux). (Built with Electron + Webrecorder)☆446Updated 4 years ago
- Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)☆160Updated 4 years ago
- NOTE: This project is no longer being actively developed.. Check out https://replayweb.page / https://github.com/webrecorder/replayweb.pa…☆201Updated 3 months ago
- Python library for reading and writing warc files☆239Updated 3 years ago
- A list of tools related to W(eb)ARC(hive)☆59Updated 10 years ago
- Tool and library for handling Web ARChive (WARC) files.☆157Updated 6 months ago
- Collect and revisit web pages.☆1,497Updated 3 months ago
- Webrecorder Desktop App!☆205Updated 4 years ago
- Web archiving using Google Chrome☆44Updated 5 years ago
- wabac.js - Web Archive Browsing Augmentation Client☆107Updated last week
- Streaming WARC/ARC library for fast web archive IO☆408Updated 4 months ago
- A list of things related to software, literature, and other content for 🕣 Memento☆97Updated 10 months ago
- The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns☆1,474Updated 9 months ago
- Rewriting web proxy and archival tool. At this point, it just tries to download all the things.☆202Updated last week
- Run a high-fidelity browser-based web archiving crawler in a single Docker container☆753Updated last week
- A search interface and wayback machine for the UKWA Solr based warc-indexer framework.☆112Updated this week
- An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed…☆149Updated 2 weeks ago
- Convert Directories, Files and ZIP Files to Web Archives (WARC)☆85Updated last month
- Serving content from a WARC☆61Updated 12 years ago
- Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.☆120Updated 3 months ago
- A simple Python wrapper for the archive.is capturing service☆200Updated 2 months ago
- WARC and ARC indexing and discovery tools.☆123Updated last month
- Convert HTTP Archive (HAR) -> Web Archive (WARC) format☆51Updated 6 years ago