ronomon / deduplication
Fast multi-threaded content-dependent chunking deduplication for Buffers in C++ with a reference implementation in Javascript. Ships with extensive tests, a fuzz test and a benchmark.
☆74Updated 4 years ago
Alternatives and similar repositories for deduplication:
Users that are interested in deduplication are comparing it to the libraries listed below
- Fast implementation of Content Defined Chunking (CDC) based on a rolling Rabin Checksum in C.☆52Updated 10 years ago
- Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.☆71Updated 2 years ago
- A not-yet-ready-for-use FoundationDB-backed FUSE filesystem. Seriously, don't use it.☆13Updated 10 months ago
- A compact implementation of Dr. Askitis HatTrie☆80Updated 10 years ago
- Append-only key-value database on a distributed shared-log☆49Updated 6 months ago
- WebR2sync+☆12Updated 7 years ago
- Rolling Hash C++ Library☆188Updated 11 months ago
- HTTP benchmark utility☆21Updated last year
- Library implementing the storage and the query evaluation for a text search engine. It uses on a key value store database interface to st…☆47Updated 3 years ago
- A simple Web Assembly JIT Engine with no dependencies☆24Updated 7 years ago
- A CPace PAKE implementation using libsodium.☆35Updated 4 years ago
- Initial work on SQLite and LMDB integration☆71Updated 2 years ago
- Fast, SIMD-accelerated hash function for content-defined chunking☆23Updated 4 years ago
- EliasFanoCompression: quasi-succinct compression of sorted integers in C#☆45Updated 3 years ago
- Automatically exported from code.google.com/p/smhasher☆16Updated 3 years ago
- Batch Monitor - Gain performance by combining work from multiple threads into a single batch☆30Updated 9 years ago
- NDB server☆66Updated 8 months ago
- fixed-length integer trim☆33Updated last year
- Replicated Block Device backed by FoundationDB☆39Updated 3 years ago
- Fast Hash Functions Using AES Intrinsics☆82Updated 5 years ago
- B-tree library for use with remote storage (DynamoDB, S3) in C++☆22Updated 7 years ago
- Official Github Mirror of the LumoSQL Database Project (https://lumosql.org/src/lumosql)☆179Updated last year
- Keys and Secrets Management Service☆56Updated 2 months ago
- The dream accurate approximate set cardinality estimator based on 3-bit HyperLogLog. More accurate than Redis HyperLogLog.☆55Updated 3 years ago
- mpool UAPI and CLI for HSE 1.x☆35Updated 3 years ago
- actordb storage engine☆19Updated 5 years ago
- a hash-based key-value database for persistent storing massive small records☆40Updated 10 years ago
- SQLite Clustered Database☆18Updated 5 years ago
- High-performance nosql database. Written in C, multi-threaded, based on LightningDB and nanomsg.☆116Updated 11 years ago
- ☆53Updated 6 years ago