ArchiveTeam / grab-siteLinks
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
☆1,542Updated 6 months ago
Alternatives and similar repositories for grab-site
Users that are interested in grab-site are comparing it to the libraries listed below
Sorting:
- Wget-compatible web downloader and crawler.☆595Updated last year
- Webrecorder Player for Desktop (OSX/Windows/Linux). (Built with Electron + Webrecorder)☆446Updated 5 years ago
- Collect and revisit web pages.☆1,525Updated 10 months ago
- Core Python Web Archiving Toolkit for replay and recording of web archives☆1,585Updated this week
- Run a high-fidelity browser-based web archiving crawler in a single Docker container☆923Updated this week
- brozzler - distributed browser-based web crawler☆760Updated last week
- Web Archiving Integration Layer: One-Click User Instigated Preservation☆383Updated 8 months ago
- Use yt-dlp to download video/metadata and upload to the Internet Archive.☆468Updated 2 weeks ago
- WARC writing MITM HTTP/S proxy☆429Updated last month
- A Python and Command-Line Interface to Archive.org☆1,790Updated last week
- Serverless replay of web archives directly in the browser☆861Updated this week
- Indexes open directories☆1,283Updated last week
- An archiving tool with an IM-style interface that prioritizes privacy and accessibility, integrated with various archival services includ…☆2,099Updated last week
- List of data-hoarding related tools☆1,244Updated 2 years ago
- Self-Hosted Bookmark And Archive Manager☆1,828Updated last year
- Chrome extension to "Create WARC files from any webpage"☆224Updated last year
- Browsertrix is the hosted, high-fidelity, browser-based crawling service from Webrecorder designed to make web archiving easier and more …☆356Updated this week
- Starting point for archiving entire YouTube channels using yt-dlp (originally youtube-dl)☆509Updated last month
- Utilities for dealing with Tumblr blogs, Tumblr backup☆689Updated 9 months ago
- Download the entire Wayback Machine archive for a given URL.☆3,104Updated 7 months ago
- Web Extension to save a faithful copy of an entire web page in a self-extracting ZIP file☆1,912Updated 3 months ago
- A curated list of awesome tools for website diffing and change monitoring.☆511Updated last month
- Webrecorder Desktop App!☆205Updated 4 years ago
- find duplicate files utility☆1,195Updated 2 months ago
- Extremely fast tool to remove duplicates and other lint from your filesystem☆2,214Updated 2 months ago
- 💾 dn - offline full-text search and archiving for your Chromium-based browser.☆3,858Updated this week
- A filesystem which allows you to mount HTTP directory listings or a single file, with a permanent cache. Now with Airsonic / Subsonic sup…☆835Updated 2 months ago
- Indelible links☆489Updated last week
- Intelligent Bookmarking☆265Updated 2 years ago
- 😇 A Docker Compose bundle to run on servers with spare CPU, RAM, disk, and bandwidth to help the world. Includes Tor, ArchiveWarrior, B…☆381Updated 6 months ago