Alternatives and similar repositories for sandcrawler

Users that are interested in sandcrawler are comparing it to the libraries listed below

• nlnwa / warchaeology
  Command line tool for digging into WARC files
  ☆47Updated this week
• nla / outbackcdx
  Web archive index server based on RocksDB
  ☆36Updated last month
• oduwsdl / MemGator
  A Memento Aggregator CLI and Server in Go
  ☆71Updated 8 months ago
• internetarchive / fatcat
  Perpetual Access To The Scholarly Record
  ☆120Updated last year
• webrecorder / cdxj-indexer
  CDXJ Indexing of WARC/ARCs
  ☆30Updated 11 months ago
• datatogether / warc
  Golang WARC (Web ARChive) Library
  ☆30Updated 6 years ago
• internetarchive / fatcat-scholar
  search interface for scholarly works
  ☆85Updated last year
• webrecorder / browsertrix-behaviors
  Automated behaviors that run in browser to interact with complex sites automatically. Used by ArchiveWeb.page and Browsertrix Crawler.
  ☆52Updated this week
• webrecorder / har2warc
  Convert HTTP Archive (HAR) -> Web Archive (WARC) format
  ☆54Updated 7 years ago
• edsu / memento-cli
  A command line utility for listing and searching snapshots in web archives
  ☆17Updated last year
• ikreymer / webarchive-indexing
  Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
  ☆46Updated 7 years ago
• iipc / warc2html
  Converts WARC files to static HTML
  ☆49Updated 2 months ago
• harvard-lil / WARC-diff-tools
  Comparing warc files
  ☆17Updated 6 years ago
• internetarchive / trough
  Trough: Big data, small databases.
  ☆40Updated last year
• nichtich / wdq
  Command line interface to Wikidata Query Service
  ☆55Updated last year
• siznax / khnum
  The god of human readable numbers
  ☆13Updated 6 years ago
• archivesunleashed / notebooks
  Various examples of notebooks for working with web archives with the Archives Unleashed Toolkit, and derivatives generated by the Archive…
  ☆26Updated 2 years ago
• Wikidata / soweego
  Link Wikidata items to large catalogs
  ☆96Updated last month
• LibraryCarpentry / lc-shell
  Library Carpentry: The UNIX Shell
  ☆32Updated this week
• steffenfritz / FileTrove
  FileTrove indexes files and creates metadata from them.
  ☆49Updated 6 months ago
• mementoweb / timegate
  A Memento TimeGate
  ☆44Updated 5 years ago
• alard / megawarc
  Nondestructive warc-in-tar to warc conversion
  ☆27Updated 12 years ago
• iipc / warc-specifications
  Centralised repository for WARC usage specifications.
  ☆119Updated last month
• magnusmanske / papers
  ☆11Updated 5 months ago
• archivesunleashed / aut
  The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
  ☆147Updated last year
• dlenski / wp2git
  Downloads and imports Wikipedia page histories to a git repository
  ☆35Updated 2 weeks ago
• ukwa / webarchive-discovery
  Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in t…
  ☆131Updated last week
• internetarchive / pdf_trio
  A PDF classifier ensemble with REST API service
  ☆23Updated 4 years ago
• openzim / warc2zim
  Command line tool to convert a file in the WARC format to a file in the ZIM format
  ☆75Updated 8 months ago
• anjackson / sliver
  A tool for collection archival slivers of the web and web archives
  ☆16Updated 9 months ago