internetarchive / CDX-WriterLinks
Python script to create CDX index files of WARC data
☆21Updated 4 months ago
Alternatives and similar repositories for CDX-Writer
Users that are interested in CDX-Writer are comparing it to the libraries listed below
Sorting:
- Web archiving using Google Chrome☆46Updated 6 years ago
- Convert HTTP Archive (HAR) -> Web Archive (WARC) format☆55Updated 7 years ago
- craigslist blob service☆92Updated 8 years ago
- Wikipedia citation tool for Google Books, New York Times, ISBN, DOI and more☆22Updated 9 years ago
- Check out https://github.com/webrecorder/webrecorder for newer version matching https://webrecorder.io☆38Updated 10 years ago
- Nondestructive warc-in-tar to warc conversion☆27Updated 12 years ago
- React components to render differences between captures at the Wayback Machine☆37Updated last week
- INACTIVE - Service powering snippets on Firefox's about:home.☆31Updated 11 months ago
- A list of things related to software, literature, and other content for 🕣 Memento☆104Updated 3 weeks ago
- URLTeam's second generation of URL shortener archiving tools☆81Updated 4 months ago
- Grabbing all news.☆61Updated 6 years ago
- Chrome extension that uses Memento to indicate that a page a user is viewing on the live web has an archived copy and to give the user ac…☆57Updated 5 months ago
- Making a reusable toolkit for writing seesaw scripts☆73Updated last week
- Specialised bot for periodical grabs and video/audio/etc. webpage scrapes.☆11Updated 8 years ago
- Datasette plugin that adds a .atom output format☆14Updated 2 months ago
- Serving content from a WARC☆62Updated 13 years ago
- A Twitter bot that archives tweets on demand.☆27Updated 7 years ago
- Archive.org OPDS Bookserver - A standard for digital book distribution☆130Updated 7 years ago
- A Memento Client Library in Python☆26Updated 7 years ago
- A command line tool to archive a git repository from GitHub to the Internet Archive.☆92Updated 4 years ago
- Decentralized web archiving☆20Updated 7 years ago
- WarcMiddleware lets users seamlessly download a mirror copy of a website when running a web crawl with the Python web crawler Scrapy.☆48Updated 7 years ago
- A validator for syndicated feeds. It works with Atom, RSS feeds as well as OPML and KML formats.☆120Updated 2 years ago
- Web-based whois gateway written in Python for lighttpd☆26Updated last year
- 🔑 Drop-in OAuth client flows for Python on Google App Engine.☆39Updated this week
- A queue-controlled browser automation tool for improving web crawl quality☆64Updated 5 months ago
- IRC bot that is being used on number of wikimedia channels☆37Updated 2 years ago
- An extension to help curate a dataset of pages that show in-page pop-ups☆12Updated 7 years ago
- Multi-platform Docker container with utilities to process images (imagemagick, exiftool, optipng...).☆13Updated last week
- A dockerized, queued high fidelity web archiver based on Squidwarc☆60Updated last year