Alternatives and similar repositories for deduplication

Users that are interested in deduplication are comparing it to the libraries listed below

• LumoSQL / archive1-LumoSQL-on-github
  Initial work on SQLite and LMDB integration
  ☆78Updated 2 years ago
• RaphaelJ / fast_strstr
  A fast substitution to the stdlib's strstr() sub-string search function.
  ☆117Updated 10 years ago
• rgov / sqlite_protobuf
  A SQLite extension for extracting values from serialized Protobuf messages
  ☆87Updated last month
• willemt / bmon
  Batch Monitor - Gain performance by combining work from multiple threads into a single batch
  ☆30Updated 10 years ago
• patrickfrey / strus
  Library implementing the storage and the query evaluation for a text search engine. It uses on a key value store database interface to st…
  ☆47Updated 3 years ago
• pmwkaa / engine.so
  Tracking, Benchmarking and Sharing Information about an open source embedded data storage engines, internals, architectures, data storage…
  ☆182Updated 6 years ago
• techrah / sqlite3-compression-encryption-vfs
  Compression and Encryption Virtual File System for SQLite 3.
  ☆118Updated 4 years ago
• gtoubassi / femtozip
  FemtoZip is a "shared dictionary" compression library optimized for small documents that may not compress well with traditional tools suc…
  ☆145Updated 4 years ago
• malbrain / HatTrie
  A compact implementation of Dr. Askitis HatTrie
  ☆80Updated 11 years ago
• backtrace-labs / umash
  UMASH: a fast enough hash and fingerprint with collision bounds
  ☆157Updated 6 months ago
• LumoSQL / lumosql
  Official Github Mirror of the LumoSQL Database Project (https://lumosql.org/src/lumosql)
  ☆203Updated 2 years ago
• jedisct1 / cpace
  A CPace PAKE implementation using libsodium.
  ☆36Updated 4 years ago
• cruzdb / cruzdb
  Append-only key-value database on a distributed shared-log
  ☆52Updated 11 months ago
• fidlej / idzip
  Automatically exported from code.google.com/p/idzip
  ☆40Updated 10 years ago
• ottypes / libot
  A minimal C library for operational transform
  ☆110Updated 10 years ago
• WebDeltaSync / WebR2sync_plus
  WebR2sync+
  ☆12Updated 8 years ago
• couchbase / fleece
  A super-fast, compact, JSON-equivalent binary data format
  ☆322Updated last week
• coreperf / rejit
  fast SIMD-able JIT regular expression compiler
  ☆196Updated 10 years ago
• backtrace-labs / stuffed-record-stream
  Backtrace.io's resilient log record framing format
  ☆38Updated 3 years ago
• ronomon / direct-io
  Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.
  ☆74Updated 2 years ago
• pascaldekloe / flit
  fixed-length integer trim
  ☆34Updated 2 years ago
• jandrewrogers / AquaHash
  Fast Hash Functions Using AES Intrinsics
  ☆82Updated 5 years ago
• LMDB / memcachedb
  memcachedb ported from BerkeleyDB to LMDB originally from http://memcachedb.googlecode.com/svn/trunk
  ☆87Updated 10 years ago
• LMDB / sqlightning
  SQLite3 ported to use LMDB instead of its original Btree code. See https://github.com/LumoSQL/LumoSQL for maintained fork.
  ☆313Updated 5 years ago
• jandrewrogers / MetroHash
  MetroHash v1: Exceptionally fast and statistically robust hash functions
  ☆370Updated 2 years ago
• symeapp / srp-client
  An RFC 5054-compatible SRP Client for Javascript.
  ☆68Updated 9 years ago
• injinj / smhasher
  Automatically exported from code.google.com/p/smhasher
  ☆16Updated 4 years ago
• fd0 / rabin-cdc
  Fast implementation of Content Defined Chunking (CDC) based on a rolling Rabin Checksum in C.
  ☆55Updated 10 years ago
• Amaury / FineDB
  High-performance nosql database. Written in C, multi-threaded, based on LightningDB and nanomsg.
  ☆116Updated 11 years ago
• jkominek / fdbfs
  A not-yet-ready-for-use FoundationDB-backed FUSE filesystem. Seriously, don't use it.
  ☆14Updated last year