internetarchive / Zeno
State-of-the-art web crawler 🔱
☆159Updated this week
Alternatives and similar repositories for Zeno
Users that are interested in Zeno are comparing it to the libraries listed below
Sorting:
- A tool for detecting viruses and NSFW material in WARC files☆14Updated 9 months ago
- Browsertrix is the hosted, high-fidelity, browser-based crawling service from Webrecorder designed to make web archiving easier and more …☆273Updated this week
- Experimental proxy and wrapper for safely embedding Web Archives (warc, warc.gz, wacz) into web pages.☆31Updated last week
- Command line tool for digging into WARC files☆40Updated this week
- A Memento Aggregator CLI and Server in Go☆64Updated 2 months ago
- wabac.js - Web Archive Browsing Augmentation Client☆107Updated this week
- 🍨 High-fidelity, browser-based, single-page web archiving library and CLI for witnessing the web.☆159Updated 2 weeks ago
- Specifications developed and maintained by the Webrecorder community.☆131Updated 4 months ago
- Web archive index server based on RocksDB☆34Updated last week
- ☆16Updated last month
- A tool for collection archival slivers of the web and web archives☆13Updated 3 months ago
- Centralised repository for WARC usage specifications.☆109Updated 5 months ago
- Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.☆46Updated 7 years ago
- Command line tool to convert a file in the WARC format to a file in the ZIM format☆57Updated last month
- Digital Preservation of HTTP in documentary heritage.☆22Updated last year
- Static Site Generator for Viewing Web Archives (in WACZ) format☆26Updated last year
- JavaScript module and CLI tool for working with web archive data using the WACZ format specification.☆14Updated 2 months ago
- CDXJ Indexing of WARC/ARCs☆25Updated 5 months ago
- Converts WARC files to static HTML☆44Updated 10 months ago
- A search interface and wayback machine for the UKWA Solr based warc-indexer framework.☆113Updated this week
- search interface for scholarly works☆85Updated 9 months ago
- WARC + AI - Experimental Retrieval Augmented Generation Pipeline for Web Archive Collections.☆248Updated 3 months ago
- A list of things related to software, literature, and other content for 🕣 Memento☆97Updated 11 months ago
- Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.☆120Updated 4 months ago
- Wombat.js client-side rewriting library☆93Updated 2 weeks ago
- Automated behaviors that run in browser to interact with complex sites automatically. Used by ArchiveWeb.page and Browsertrix Crawler.☆40Updated 2 weeks ago
- Support for writing WARC files with Scrapy☆21Updated 5 years ago
- A polite and user-friendly downloader for Common Crawl data☆43Updated last week
- 📚 A compilation of research relevant to Data Together's efforts tackling the general problem of data resilience & interactivity☆95Updated 6 years ago
- brozzler - distributed browser-based web crawler☆708Updated this week