ronomon / deduplication
Fast multi-threaded content-dependent chunking deduplication for Buffers in C++ with a reference implementation in Javascript. Ships with extensive tests, a fuzz test and a benchmark.
☆72Updated 4 years ago
Related projects ⓘ
Alternatives and complementary repositories for deduplication
- A SQLite extension for extracting values from serialized Protobuf messages☆87Updated 2 years ago
- Fast Hash Functions Using AES Intrinsics☆82Updated 5 years ago
- Append-only key-value database on a distributed shared-log☆49Updated 3 months ago
- Batch Monitor - Gain performance by combining work from multiple threads into a single batch☆31Updated 9 years ago
- Fast implementation of Content Defined Chunking (CDC) based on a rolling Rabin Checksum in C.☆49Updated 10 years ago
- MergedTrie code☆12Updated 4 years ago
- A compact implementation of Dr. Askitis HatTrie☆80Updated 10 years ago
- The dream accurate approximate set cardinality estimator based on 3-bit HyperLogLog. More accurate than Redis HyperLogLog.☆55Updated 3 years ago
- Initial work on SQLite and LMDB integration☆68Updated last year
- A CPace PAKE implementation using libsodium.☆35Updated 3 years ago
- HTTP benchmark utility☆21Updated last year
- a quotient filter written in C☆84Updated 7 years ago
- NuDB: A fast key/value insert-only database for SSD drives in C++11☆28Updated 5 years ago
- Automatically exported from code.google.com/p/idzip☆40Updated 9 years ago
- Tracking, Benchmarking and Sharing Information about an open source embedded data storage engines, internals, architectures, data storage…☆180Updated 5 years ago
- EliasFanoCompression: quasi-succinct compression of sorted integers in C#☆42Updated 3 years ago
- Bloom filter alternative (C++)☆17Updated 6 years ago
- B-tree library for use with remote storage (DynamoDB, S3) in C++☆23Updated 7 years ago
- Official Github Mirror of the LumoSQL Database Project (https://lumosql.org/src/lumosql)☆178Updated last year
- a hash-based key-value database for persistent storing massive small records☆41Updated 10 years ago
- Library implementing the storage and the query evaluation for a text search engine. It uses on a key value store database interface to st…☆47Updated 3 years ago
- Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.☆69Updated last year
- An inverted trigram index for accelerated string matching in Sqlite.☆77Updated 10 years ago
- A Wait-Free Universal Construct for Large Objects☆96Updated 4 years ago
- a 64-bit histogram / quantile sketch☆56Updated last year
- Assorted notes☆82Updated 10 months ago
- FemtoZip is a "shared dictionary" compression library optimized for small documents that may not compress well with traditional tools suc…☆146Updated 3 years ago
- WebR2sync+☆12Updated 7 years ago
- A fast substitution to the stdlib's strstr() sub-string search function.☆116Updated 9 years ago
- mpool UAPI and CLI for HSE 1.x☆35Updated 3 years ago