ArchiveTeam / wget-lua
Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
☆108Updated 3 weeks ago
Alternatives and similar repositories for wget-lua:
Users that are interested in wget-lua are comparing it to the libraries listed below
- Command line tool to convert a file in the WARC format to a file in the ZIM format☆49Updated 2 weeks ago
- 🎭 An introduction to the Internet Archiving ecosystem, tooling, and some of the ethical dilemmas that the community faces.☆52Updated 5 months ago
- Encode/decode binary data over a live streaming video in real time.☆13Updated last year
- Convert HTTP Archive (HAR) -> Web Archive (WARC) format☆51Updated 6 years ago
- Chrome extension to "Create WARC files from any webpage"☆216Updated last year
- Nondestructive warc-in-tar to warc conversion☆26Updated 11 years ago
- Conifer setup and deployment via Ansible☆12Updated 4 years ago
- Recover lost websites from the Web Infrastructure☆87Updated 3 years ago
- Scripts to build and boot warrior virtual machine containing Docker☆114Updated last year
- Docker Compose based system for running remote browsers (including Flash and Java support) connected to web archives☆14Updated 3 years ago
- URLTeam's second generation of URL shortener archiving tools☆74Updated 2 weeks ago
- 🍨 High-fidelity, browser-based, single-page web archiving library and CLI for witnessing the web.☆126Updated last week
- Homebrew formula for the ArchiveBox self-hosted internet archiving solution.☆28Updated 3 months ago
- wpull fork with fixes and faster parsing using html5-parser; used by grab-site; should go away when wpull is similarly improved☆27Updated 6 months ago
- DigestBox takes any webpage URL (news article, video link, comment thread, etc.) and gives you just the raw content. It's powered by Arch…☆18Updated 11 months ago
- A command line tool to archive a git repository from GitHub to the Internet Archive.☆92Updated 3 years ago
- Automated behaviors that run in browser to interact with complex sites automatically. Used by ArchiveWeb.page and Browsertrix Crawler.☆37Updated 2 weeks ago
- Read and write WARC files in Go☆21Updated this week
- WIP - scripts for analyzing the (in)security of Chrome extensions☆26Updated 10 months ago
- simple script to convert web resources to a single warc file☆19Updated last year
- Web archiving using Google Chrome☆44Updated 5 years ago
- Web Archiving Integration Layer: One-Click User Instigated Preservation☆356Updated 3 months ago
- The ArchiveWeb.page Site☆28Updated last month
- Finger.Farm Modern Finger Protocol Hosting... kind of a fingerd implementation in Node☆44Updated 3 years ago
- Clean a series of links, resolving redirects and finding Wayback results if page is gone. Originally written to aid with importing from A…☆16Updated 3 months ago
- 🎭 An introduction to the Internet Archiving ecosystem, tooling, and some of the ethical dilemmas that the community faces.☆14Updated 4 years ago
- Music Of a People. Music for the people by the people. #BuildPublicDomain☆14Updated 6 months ago
- Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head☆171Updated 4 years ago
- Wombat.js client-side rewriting library☆88Updated 2 months ago
- Links on the web break all the time, robustify them!☆52Updated 4 years ago