Alternatives and similar repositories for deduplication

Users that are interested in deduplication are comparing it to the libraries listed below

• fd0 / rabin-cdc
  Fast implementation of Content Defined Chunking (CDC) based on a rolling Rabin Checksum in C.
  ☆56Updated 10 years ago
• nlfiedler / fastcdc-rs
  FastCDC implementation in Rust
  ☆156Updated last month
• WebDeltaSync / WebR2sync_plus
  WebR2sync+
  ☆12Updated 8 years ago
• cruzdb / cruzdb
  Append-only key-value database on a distributed shared-log
  ☆49Updated 9 months ago
• hse-project / mpool
  mpool UAPI and CLI for HSE 1.x
  ☆36Updated 3 years ago
• rix0rrr / libbruce
  B-tree library for use with remote storage (DynamoDB, S3) in C++
  ☆22Updated 7 years ago
• Alex-Werner / TyrDB
  Fast in-memory database for node and browser
  ☆27Updated 11 months ago
• malbrain / HatTrie
  A compact implementation of Dr. Askitis HatTrie
  ☆80Updated 11 years ago
• jkominek / fdbfs
  A not-yet-ready-for-use FoundationDB-backed FUSE filesystem. Seriously, don't use it.
  ☆13Updated last year
• biokoda / actordb_driver
  actordb storage engine
  ☆20Updated 5 years ago
• wangyi-fudan / wyHLL
  The dream accurate approximate set cardinality estimator based on 3-bit HyperLogLog. More accurate than Redis HyperLogLog.
  ☆55Updated 4 years ago
• ronomon / direct-io
  Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.
  ☆74Updated 2 years ago
• willemt / bmon
  Batch Monitor - Gain performance by combining work from multiple threads into a single batch
  ☆30Updated 10 years ago
• jandrewrogers / AquaHash
  Fast Hash Functions Using AES Intrinsics
  ☆82Updated 5 years ago
• LumoSQL / archive1-LumoSQL-on-github
  Initial work on SQLite and LMDB integration
  ☆73Updated 2 years ago
• gate-computer / gate
  Benign remote code execution
  ☆111Updated 2 months ago
• cybertec-postgresql / zheap
  ⚡️ Development status and progress reporting.
  ☆92Updated 3 months ago
• mikeowens / SQLiteVTable
  SQLite Virtual Table Example
  ☆25Updated 9 years ago
• farsightsec / mtbl
  immutable sorted string table library
  ☆80Updated 4 months ago
• srijs / rust-gearhash
  Fast, SIMD-accelerated hash function for content-defined chunking
  ☆24Updated 5 years ago
• jsnell / parallel-xxhash
  Compute xxHash hash codes for 8 keys in parallel
  ☆46Updated 6 years ago
• backtrace-labs / stuffed-record-stream
  Backtrace.io's resilient log record framing format
  ☆38Updated 3 years ago
• pascaldekloe / flit
  fixed-length integer trim
  ☆34Updated 2 years ago
• 2ndQuadrant / fixeddecimal
  PostgreSQL data type for efficient storage of monetary values. Much faster performance than NUMERIC. Works with Postgres-XL also.
  ☆33Updated 5 years ago
• rgov / sqlite_protobuf
  A SQLite extension for extracting values from serialized Protobuf messages
  ☆88Updated 3 months ago
• LumoSQL / lumosql
  Official Github Mirror of the LumoSQL Database Project (https://lumosql.org/src/lumosql)
  ☆196Updated last year
• coreperf / rejit
  fast SIMD-able JIT regular expression compiler
  ☆193Updated 10 years ago
• sachin-sinha / BangDB
  BangDB - nosql database
  ☆55Updated 2 months ago
• LMDB / memcachedb
  memcachedb ported from BerkeleyDB to LMDB originally from http://memcachedb.googlecode.com/svn/trunk
  ☆87Updated 10 years ago
• vidardb / pgrocks-fdw
  Bring RocksDB to PostgreSQL as an extension. It is the first foreign data wrapper (FDW) that introduces LSM-tree into PostgreSQL. The und…
  ☆128Updated 2 years ago