Alternatives and similar repositories for deduplication

Users that are interested in deduplication are comparing it to the libraries listed below

• fd0 / rabin-cdc
  Fast implementation of Content Defined Chunking (CDC) based on a rolling Rabin Checksum in C.
  ☆56Updated 10 years ago
• fidlej / idzip
  Automatically exported from code.google.com/p/idzip
  ☆40Updated 10 years ago
• malbrain / HatTrie
  A compact implementation of Dr. Askitis HatTrie
  ☆80Updated 11 years ago
• cruzdb / cruzdb
  Append-only key-value database on a distributed shared-log
  ☆49Updated 10 months ago
• nlfiedler / fastcdc-rs
  FastCDC implementation in Rust
  ☆158Updated 2 months ago
• fredrikwidlund / pounce
  HTTP benchmark utility
  ☆23Updated 2 years ago
• rix0rrr / libbruce
  B-tree library for use with remote storage (DynamoDB, S3) in C++
  ☆22Updated 7 years ago
• rgov / sqlite_protobuf
  A SQLite extension for extracting values from serialized Protobuf messages
  ☆88Updated last week
• LumoSQL / lumosql
  Official Github Mirror of the LumoSQL Database Project (https://lumosql.org/src/lumosql)
  ☆198Updated last year
• willemt / bmon
  Batch Monitor - Gain performance by combining work from multiple threads into a single batch
  ☆30Updated 10 years ago
• fusiyuan2010 / cuttdb
  a hash-based key-value database for persistent storing massive small records
  ☆40Updated 11 years ago
• pascaldekloe / flit
  fixed-length integer trim
  ☆34Updated 2 years ago
• rcoh / treelayout
  Demonstrating the importance of laying out data in memory.
  ☆47Updated 7 years ago
• jandrewrogers / AquaHash
  Fast Hash Functions Using AES Intrinsics
  ☆82Updated 5 years ago
• biokoda / actordb_driver
  actordb storage engine
  ☆20Updated 5 years ago
• patrickfrey / strus
  Library implementing the storage and the query evaluation for a text search engine. It uses on a key value store database interface to st…
  ☆47Updated 3 years ago
• vedantk / quotient-filter
  a quotient filter written in C
  ☆84Updated 7 years ago
• JohnSully / libwasm
  A simple Web Assembly JIT Engine with no dependencies
  ☆26Updated 7 years ago
• tomerdmnt / levelfs
  leveldb FUSE filesystem
  ☆16Updated 11 years ago
• jsnell / parallel-xxhash
  Compute xxHash hash codes for 8 keys in parallel
  ☆46Updated 6 years ago
• KioxiaAmerica / trocksdb
  RocksDB modified to separate keys and values, resulting in a major decrease in write amplification without sacrificing performance
  ☆55Updated 4 years ago
• rasky / gcs
  Golomb Coded Sets
  ☆94Updated 8 years ago
• xtreemfs / babudb
  An embedded non-relational database for Java and C++
  ☆37Updated 9 months ago
• hse-project / mpool
  mpool UAPI and CLI for HSE 1.x
  ☆36Updated 3 years ago
• fwessels / simdcsv
  ☆36Updated 4 years ago
• wolfgarbe / EliasFanoCompression
  EliasFanoCompression: quasi-succinct compression of sorted integers in C#
  ☆45Updated 3 years ago
• fabianlindfors / fdb-object-store
  Simple object store built on top of FoundationDB
  ☆22Updated 7 years ago
• ronomon / direct-io
  Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.
  ☆74Updated 2 years ago
• tidwall / rocksdb-server
  Fast Redis clone written in C using RocksDB as a backend.
  ☆75Updated 3 years ago
• wangyi-fudan / wyHLL
  The dream accurate approximate set cardinality estimator based on 3-bit HyperLogLog. More accurate than Redis HyperLogLog.
  ☆55Updated 4 years ago