openzim / python-scraperlibLinks
Collection of Python code to re-use across Python-based scrapers
☆25Updated last week
Alternatives and similar repositories for python-scraperlib
Users that are interested in python-scraperlib are comparing it to the libraries listed below
Sorting:
- Command line tool to convert a file in the WARC format to a file in the ZIM format☆75Updated 2 weeks ago
- 🎭 An introduction to the Internet Archiving ecosystem, tooling, and some of the ethical dilemmas that the community faces.☆58Updated last year
- 🎭 An introduction to the Internet Archiving ecosystem, tooling, and some of the ethical dilemmas that the community faces.☆15Updated 5 years ago
- Libzim binding for Python: read/write ZIM files in Python☆97Updated last month
- Turns a collection of documents into a browsable ZIM file☆26Updated last month
- Clean a series of links, resolving redirects and finding Wayback results if page is gone. Originally written to aid with importing from A…☆19Updated 2 months ago
- DigestBox takes any webpage URL (news article, video link, comment thread, etc.) and gives you just the raw content. It's powered by Arch…☆19Updated 2 years ago
- ActivityPub server without Javascript, designed for simplicity and accessibility. Includes calendar, news and sharing economy features to…☆75Updated last week
- Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.☆131Updated 3 weeks ago
- A list of things related to software, literature, and other content for 🕣 Memento☆104Updated 3 weeks ago
- Automated behaviors that run in browser to interact with complex sites automatically. Used by ArchiveWeb.page and Browsertrix Crawler.☆55Updated this week
- Convert an online sitemap to Atom, RSS and JSON feeds☆61Updated 2 years ago
- anagora.org/node/agora-bot☆23Updated last week
- Tool to index and serve HTML files. Powered by Datasette.☆111Updated 3 years ago
- Passively capture, archive, and hoard your web browsing history, including the contents of the pages you visit, for later offline viewing…☆109Updated 3 months ago
- Agora Server implementation for the Agora (flancia.org/go/agora).☆63Updated last week
- Awesome links related to RSS, ATOM, and Syndication formats.☆61Updated last year
- This is the HeadQuarters of my digital info. HPI library got me inspired and I'm trying to play with the idea on a smaller scale for myse…☆21Updated 2 years ago
- Various ZIM command line tools☆189Updated 3 months ago
- Generic automation tool around data stored as plaintext YAML files☆37Updated this week
- Kiwix & openZIM build engine☆116Updated last week
- Javascript/Node wrapper around Mozilla's Readability library so that ArchiveBox can call it as a oneshot CLI command to extract each page…☆42Updated last year
- Kiwix Catalog BitTorrent Seeder Companion☆15Updated last month
- Chrome extension that uses Memento to indicate that a page a user is viewing on the live web has an archived copy and to give the user ac…☆58Updated 5 months ago
- RSS feeds in public.☆15Updated last week
- React components to render differences between captures at the Wayback Machine☆37Updated 2 weeks ago
- anonymous CLI for reading microblogging (chiefly Mastodon) posts☆19Updated 3 months ago
- Export your Github activity: events, repositories, stars, etc.☆56Updated this week
- Homebrew formula for the ArchiveBox self-hosted internet archiving solution.☆28Updated last year
- A pinboard for audiovisual media.☆25Updated 3 months ago