Alternatives and similar repositories for qwarc

Users that are interested in qwarc are comparing it to the libraries listed below

• ArchiveTeam / wget-lua
  Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
  ☆130Updated 3 months ago
• ArchiveTeam / ludios_wpull
  wpull fork with fixes and faster parsing using html5-parser; used by grab-site; should go away when wpull is similarly improved
  ☆30Updated 3 months ago
• webrecorder / browsertrix-behaviors
  Automated behaviors that run in browser to interact with complex sites automatically. Used by ArchiveWeb.page and Browsertrix Crawler.
  ☆53Updated 3 weeks ago
• gdamdam / iagitup
  A command line tool to archive a git repository from GitHub to the Internet Archive.
  ☆92Updated 4 years ago
• openzim / warc2zim
  Command line tool to convert a file in the WARC format to a file in the ZIM format
  ☆75Updated this week
• internetarchive / scrapy-warcio
  Support for writing WARC files with Scrapy
  ☆24Updated 6 years ago
• chfoo / warcat
  Tool and library for handling Web ARChive (WARC) files.
  ☆165Updated last year
• mediawiki-client-tools / mediawiki-dump-generator
  Python 3 tools for downloading and preserving wikis
  ☆123Updated 5 months ago
• webrecorder / wombat
  Wombat.js client-side rewriting library
  ☆108Updated 2 weeks ago
• ArchiveTeam / universal-tracker
  A configurable, reusable tracker with dashboard
  ☆36Updated 2 years ago
• medialab / ural
  A helper library full of URL-related heuristics.
  ☆73Updated 2 months ago
• peterk / warcworker
  A dockerized, queued high fidelity web archiver based on Squidwarc
  ☆61Updated last year
• zytedata / web-snap
  Create "perfect" snapshots of web pages
  ☆33Updated this week
• pirate / internet-archiving-talk
  🎭 An introduction to the Internet Archiving ecosystem, tooling, and some of the ethical dilemmas that the community faces.
  ☆58Updated last year
• JustAnotherArchivist / little-things
  The little things give you away... A collection of various small helper stuff – Mirror repo only, no longer kept in sync, refer to gitea.…
  ☆24Updated 5 years ago
• mikwielgus / forum-dl
  Scrape posts, threads from forums, news aggregators, mail archives, export to JSONL, mailbox, WARC
  ☆105Updated last year
• harvard-lil / scoop
  🍨 High-fidelity, browser-based, single-page web archiving library and CLI for witnessing the web.
  ☆181Updated 3 months ago
• commoncrawl / cc-downloader
  A polite and user-friendly downloader for Common Crawl data
  ☆63Updated 4 months ago
• ArchiveTeam / yahooanswers-grab
  Saving all questions and answers from Yahoo! Answers.
  ☆50Updated 4 years ago
• ArchiveTeam / Ubuntu-Warrior
  Scripts to build and boot warrior virtual machine containing Docker
  ☆122Updated 8 months ago
• oduwsdl / warrick
  Recover lost websites from the Web Infrastructure
  ☆89Updated 4 months ago
• dlenski / wp2git
  Downloads and imports Wikipedia page histories to a git repository
  ☆35Updated last month
• machawk1 / Mink
  Chrome extension that uses Memento to indicate that a page a user is viewing on the live web has an archived copy and to give the user ac…
  ☆55Updated 3 months ago
• tsdko / pullcord
  Discord archiver
  ☆65Updated 2 years ago
• cocrawler / cdx_toolkit
  A toolkit for CDX indices such as Common Crawl and the Internet Archive's Wayback Machine
  ☆195Updated last month
• openzim / python-libzim
  Libzim binding for Python: read/write ZIM files in Python
  ☆96Updated last week
• karlb / sqlite-spellfix
  Loadable spellfix1 extension for sqlite as python package
  ☆26Updated last year
• neocities / python-neocities
  Python API for neocities.org
  ☆86Updated last year
• ArchiveTeam / imgur-grab
  Archiving imgur.
  ☆62Updated 4 months ago
• jortage / poolmgr
  [Moved to Forgejo] An S3-to-S3 proxy (and more) implementing file-level deduplication and access control.
  ☆27Updated 4 months ago