ArchiveTeam / grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
☆1,479Updated 10 months ago
Alternatives and similar repositories for grab-site:
Users that are interested in grab-site are comparing it to the libraries listed below
- Wget-compatible web downloader and crawler.☆583Updated last year
- Webrecorder Player for Desktop (OSX/Windows/Linux). (Built with Electron + Webrecorder)☆446Updated 4 years ago
- Collect and revisit web pages.☆1,500Updated 3 months ago
- Core Python Web Archiving Toolkit for replay and recording of web archives☆1,497Updated this week
- An Awesome List for getting started with web archiving☆2,237Updated 3 weeks ago
- Use yt-dlp to download video/metadata and upload to the Internet Archive.☆446Updated last month
- ArchiveBot, an IRC bot for archiving websites☆382Updated last month
- brozzler - distributed browser-based web crawler☆704Updated 3 weeks ago
- Web Archiving Integration Layer: One-Click User Instigated Preservation☆374Updated last month
- A Python and Command-Line Interface to Archive.org☆1,701Updated last week
- Run a high-fidelity browser-based web archiving crawler in a single Docker container☆767Updated this week
- An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services includ…☆1,953Updated this week
- Serverless replay of web archives directly in the browser☆788Updated this week
- A Tool To Push Web Resources Into Web Archives☆420Updated last year
- WARC writing MITM HTTP/S proxy☆402Updated last month
- Tools for downloading and preserving wikis. We archive wikis, from Wikipedia to tiniest wikis. As of 2025, WikiTeam has preserved more th…☆763Updated last month
- 💾 dn - offline full-text search and archiving for your Chromium-based browser.☆3,836Updated 2 months ago
- IA's public Wayback Machine (moved from SourceForge)☆785Updated last year
- Self-Hosted Bookmark And Archive Manager☆1,810Updated last year
- Chrome extension to "Create WARC files from any webpage"☆220Updated last year
- Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matc…☆722Updated 2 months ago
- Web Extension to save a faithful copy of an entire web page in a self-extracting ZIP file☆1,871Updated 7 months ago
- The ultimate collection of scripts for YouTube-DL.☆2,408Updated 7 months ago
- Library of Alexandria (LoA in short) is a project that aims to collect and archive documents from the internet.☆114Updated 10 months ago
- Browser extension for viewing archived and cached versions of web pages, available for Chrome, Edge and Safari☆1,312Updated 4 months ago
- I consume the world via RSS feeds, and this is my attempt to keep it that way.☆797Updated last month
- A filesystem which allows you to mount HTTP directory listings or a single file, with a permanent cache. Now with Airsonic / Subsonic sup…☆809Updated last week
- Finds larger/original versions of images and videos☆1,244Updated this week
- Diskover Community Edition - Open source file indexer, file search engine and data management and analytics powered by Elasticsearch☆1,566Updated 3 weeks ago
- A curated list of awesome tools for website diffing and change monitoring.☆508Updated 2 years ago