ronomon / deduplicationLinks
Fast multi-threaded content-dependent chunking deduplication for Buffers in C++ with a reference implementation in Javascript. Ships with extensive tests, a fuzz test and a benchmark.
☆76Updated 5 years ago
Alternatives and similar repositories for deduplication
Users that are interested in deduplication are comparing it to the libraries listed below
Sorting:
- Fast implementation of Content Defined Chunking (CDC) based on a rolling Rabin Checksum in C.☆56Updated 10 years ago
- FastCDC implementation in Rust☆156Updated last month
- WebR2sync+☆12Updated 8 years ago
- Append-only key-value database on a distributed shared-log☆49Updated 9 months ago
- mpool UAPI and CLI for HSE 1.x☆36Updated 3 years ago
- B-tree library for use with remote storage (DynamoDB, S3) in C++☆22Updated 7 years ago
- Fast in-memory database for node and browser☆27Updated 11 months ago
- A compact implementation of Dr. Askitis HatTrie☆80Updated 11 years ago
- A not-yet-ready-for-use FoundationDB-backed FUSE filesystem. Seriously, don't use it.☆13Updated last year
- actordb storage engine☆20Updated 5 years ago
- The dream accurate approximate set cardinality estimator based on 3-bit HyperLogLog. More accurate than Redis HyperLogLog.☆55Updated 4 years ago
- Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.☆74Updated 2 years ago
- Batch Monitor - Gain performance by combining work from multiple threads into a single batch☆30Updated 10 years ago
- Fast Hash Functions Using AES Intrinsics☆82Updated 5 years ago
- Initial work on SQLite and LMDB integration☆73Updated 2 years ago
- Benign remote code execution☆111Updated 2 months ago
- ⚡️ Development status and progress reporting.☆92Updated 3 months ago
- SQLite Virtual Table Example☆25Updated 9 years ago
- immutable sorted string table library☆80Updated 4 months ago
- Fast, SIMD-accelerated hash function for content-defined chunking☆24Updated 5 years ago
- Compute xxHash hash codes for 8 keys in parallel☆46Updated 6 years ago
- Backtrace.io's resilient log record framing format☆38Updated 3 years ago
- fixed-length integer trim☆34Updated 2 years ago
- PostgreSQL data type for efficient storage of monetary values. Much faster performance than NUMERIC. Works with Postgres-XL also.☆33Updated 5 years ago
- A SQLite extension for extracting values from serialized Protobuf messages☆88Updated 3 months ago
- Official Github Mirror of the LumoSQL Database Project (https://lumosql.org/src/lumosql)☆196Updated last year
- fast SIMD-able JIT regular expression compiler☆193Updated 10 years ago
- BangDB - nosql database☆55Updated 2 months ago
- memcachedb ported from BerkeleyDB to LMDB originally from http://memcachedb.googlecode.com/svn/trunk☆87Updated 10 years ago
- Bring RocksDB to PostgreSQL as an extension. It is the first foreign data wrapper (FDW) that introduces LSM-tree into PostgreSQL. The und…☆128Updated 2 years ago