ronomon / deduplication
Fast multi-threaded content-dependent chunking deduplication for Buffers in C++ with a reference implementation in Javascript. Ships with extensive tests, a fuzz test and a benchmark.
☆75Updated 5 years ago
Alternatives and similar repositories for deduplication:
Users that are interested in deduplication are comparing it to the libraries listed below
- Fast implementation of Content Defined Chunking (CDC) based on a rolling Rabin Checksum in C.☆55Updated 10 years ago
- Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.☆73Updated 2 years ago
- HTTP benchmark utility☆22Updated 2 years ago
- A Parallelized Data Deduplication and Compression utility☆281Updated 3 years ago
- Erasure Code API library written in C with pluggable Erasure Code backends. Mirror of code maintained at opendev.org.☆158Updated this week
- Automatically exported from code.google.com/p/idzip☆40Updated 10 years ago
- Fast Hash Functions Using AES Intrinsics☆82Updated 5 years ago
- A compact implementation of Dr. Askitis HatTrie☆80Updated 10 years ago
- mpool UAPI and CLI for HSE 1.x☆36Updated 3 years ago
- Official Github Mirror of the LumoSQL Database Project (https://lumosql.org/src/lumosql)☆182Updated last year
- The dream accurate approximate set cardinality estimator based on 3-bit HyperLogLog. More accurate than Redis HyperLogLog.☆55Updated 4 years ago
- Magically make your Redis data smaller with Zstandard compression.☆43Updated 8 years ago
- High-performance nosql database. Written in C, multi-threaded, based on LightningDB and nanomsg.☆116Updated 11 years ago
- a hash-based key-value database for persistent storing massive small records☆40Updated 10 years ago
- Embedded storage benchmarking tool☆138Updated 2 years ago
- Append-only key-value database on a distributed shared-log☆49Updated 8 months ago
- UMASH: a fast enough hash and fingerprint with collision bounds☆155Updated 3 months ago
- fixed-length integer trim☆34Updated 2 years ago
- MetroHash v1: Exceptionally fast and statistically robust hash functions☆365Updated 2 years ago
- Automatically exported from code.google.com/p/smhasher☆16Updated 4 years ago
- a quotient filter written in C☆84Updated 7 years ago
- WebR2sync+☆12Updated 8 years ago
- Benchmark showing the we can randomly hash strings very quickly with good universality☆136Updated 3 months ago
- actordb storage engine☆20Updated 5 years ago
- A CPace PAKE implementation using libsodium.☆35Updated 4 years ago
- A simple Web Assembly JIT Engine with no dependencies☆24Updated 7 years ago
- Keys and Secrets Management Service☆56Updated 4 months ago
- A utility to make deflate dictionaries☆118Updated 6 years ago
- Rolling Hash C++ Library☆187Updated last year
- Batch Monitor - Gain performance by combining work from multiple threads into a single batch☆30Updated 10 years ago