Alternatives and similar repositories for libwarc

Users that are interested in libwarc are comparing it to the libraries listed below

• helgeho / Web2Warc
  An easy-to-use and highly customizable crawler that enables you to create your own little Web archives (WARC/CDX)
  ☆25Updated 7 years ago
• internetarchive / pdf_trio
  A PDF classifier ensemble with REST API service
  ☆23Updated 4 years ago
• ikreymer / webarchive-indexing
  Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
  ☆46Updated 7 years ago
• LanguageMachines / libfolia
  FoLiA library for C++
  ☆16Updated 3 months ago
• TeamHG-Memex / extract-html-diff
  extract difference between two html pages
  ☆32Updated 7 years ago
• queertypes / boost-bloom-filters
  A collection of generic, C++ Bloom Filter classes developed for the Boost C++ Libraries.
  ☆24Updated 8 years ago
• microsoft / QBASHER
  Inverted file indexing and retrieval optimized for short texts. Supports auto-suggest and query segment classification.
  ☆34Updated last year
• Early-Modern-OCR / hOCR-De-Noising
  code to remove "noise" from hOCR output of Tesseract OCR.
  ☆14Updated 8 years ago
• TeamHG-Memex / MaybeDont
  A component that tries to avoid downloading duplicate content
  ☆27Updated 7 years ago
• internetarchive / umbra
  A queue-controlled browser automation tool for improving web crawl quality
  ☆61Updated 2 months ago
• bloomberg / pycsvw
  A tool to read CSV files with CSVW metadata and transform them into other formats.
  ☆32Updated 6 years ago
• xkxx / algodb
  Mad (╯°□°)╯'ing
  ☆10Updated 2 years ago
• xhochy / libfuzzymatch
  C++11 library for fast fuzzy searching
  ☆14Updated 9 years ago
• alard / warc-proxy
  Serving content from a WARC
  ☆61Updated 12 years ago
• nlppln / nlppln
  NLP pipeline software using common workflow language
  ☆34Updated 6 years ago
• korpling / salt
  A powerful, tagset-independent and theory-neutral meta model and API for storing, manipulating, and representing nearly all types of ling…
  ☆15Updated 2 years ago
• alephdata / pdflib
  Binary Python bindings for poppler utils for content extraction
  ☆42Updated 4 years ago
• ikreymer / cc-index-server
  Deployment of pywb as a CommonCrawl Index Server
  ☆21Updated 7 years ago
• vinaygoel / archive-analysis
  Tools to analyze web archives
  ☆20Updated 8 years ago
• ocrplusplus / ocrplusplus
  ☆40Updated 7 years ago
• pdfliberation / knowledge
  A place to collect and share knowledge about liberating data from PDFs
  ☆54Updated 3 years ago
• jermp / tongrams_estimation
  A C++ library implementing fast language models estimation using the 1-Sort algorithm.
  ☆17Updated 2 years ago
• openvenues / openvenues
  ☆24Updated 9 years ago
• inferlink / landmark-extractor
  ☆11Updated 6 years ago
• internetarchive / trough
  Trough: Big data, small databases.
  ☆42Updated 10 months ago
• poke1024 / simtrie
  An efficient data structure for fast string similarity searches
  ☆22Updated 4 years ago
• kermitt2 / biblio_glutton_harvester
  Open Access PDF harvester
  ☆40Updated last year
• mediacloud / date_guesser
  A library to extract a publication date from a web page, along with a measure of the accuracy.
  ☆41Updated 5 years ago
• seomoz / url-cpp
  C++ bindings for url parsing and sanitization
  ☆19Updated last year
• yasmina85 / OffTopic-Detection
  This repository contains tool and collections dataset for detecting off-topic pages from Web archived collections.
  ☆18Updated 9 years ago