Alternatives and similar repositories for sandcrawler

Users that are interested in sandcrawler are comparing it to the libraries listed below

• nlnwa / warchaeology
  Command line tool for digging into WARC files
  ☆46Updated last week
• datatogether / warc
  Golang WARC (Web ARChive) Library
  ☆30Updated 6 years ago
• oduwsdl / MemGator
  A Memento Aggregator CLI and Server in Go
  ☆69Updated 6 months ago
• inveniosoftware / dojson
  Simple pythonic JSON to JSON converter.
  ☆11Updated 2 months ago
• webrecorder / har2warc
  Convert HTTP Archive (HAR) -> Web Archive (WARC) format
  ☆54Updated 6 years ago
• nla / outbackcdx
  Web archive index server based on RocksDB
  ☆35Updated 3 weeks ago
• miku / span
  Span formats.
  ☆17Updated 2 weeks ago
• ikreymer / webarchive-indexing
  Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
  ☆46Updated 7 years ago
• nlnwa / gowarcserver
  ☆16Updated 5 months ago
• internetarchive / trough
  Trough: Big data, small databases.
  ☆41Updated last year
• internetarchive / pdf_trio
  A PDF classifier ensemble with REST API service
  ☆23Updated 4 years ago
• edsu / memento-cli
  A command line utility for listing and searching snapshots in web archives
  ☆17Updated last year
• iipc / warc2html
  Converts WARC files to static HTML
  ☆48Updated last week
• internetarchive / gowarc
  Read and write WARC files in Go
  ☆37Updated this week
• siznax / khnum
  The god of human readable numbers
  ☆13Updated 5 years ago
• webrecorder / cdxj-indexer
  CDXJ Indexing of WARC/ARCs
  ☆28Updated 9 months ago
• internetarchive / warctools
  Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
  ☆164Updated last month
• archivesunleashed / docker-aut
  Docker image for the Archives Unleashed Toolkit
  ☆12Updated 2 years ago
• caltechlibrary / dataset
  dataset is a command line tool, Go package, shared library and Python package for working with JSON objects as collections
  ☆24Updated last month
• ukwa / webarchive-discovery
  Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in t…
  ☆128Updated 2 months ago
• internetarchive / fatcat
  Perpetual Access To The Scholarly Record
  ☆120Updated last year
• drkane / datasette-reconcile
  Adds a reconciliation API endpoint to Datasette, based on the Reconciliation Service API specification.
  ☆24Updated last year
• dlenski / wp2git
  Downloads and imports Wikipedia page histories to a git repository
  ☆35Updated 9 months ago
• httpreserve / httpreserve
  Digital Preservation of HTTP in documentary heritage.
  ☆22Updated 2 years ago
• richardlehane / webarchive
  golang readers for ARC and WARC webarchive formats
  ☆20Updated 2 years ago
• internetarchive / fatcat-scholar
  search interface for scholarly works
  ☆86Updated last year
• nichtich / wdq
  Command line interface to Wikidata Query Service
  ☆55Updated last year
• helgeho / ArchiveSpark
  An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed…
  ☆152Updated last month
• karlb / sqlite-spellfix
  Loadable spellfix1 extension for sqlite as python package
  ☆26Updated last year
• hectorcorrea / ldpserver
  A mini LDP Server written in Go.
  ☆11Updated 9 years ago