ronomon / deduplicationLinks
Fast multi-threaded content-dependent chunking deduplication for Buffers in C++ with a reference implementation in Javascript. Ships with extensive tests, a fuzz test and a benchmark.
☆75Updated 5 years ago
Alternatives and similar repositories for deduplication
Users that are interested in deduplication are comparing it to the libraries listed below
Sorting:
- Initial work on SQLite and LMDB integration☆81Updated 2 years ago
- Official Github Mirror of the LumoSQL Database Project (https://lumosql.org/src/lumosql)☆208Updated 2 years ago
- Batch Monitor - Gain performance by combining work from multiple threads into a single batch☆30Updated 10 years ago
- A super-fast, compact, JSON-equivalent binary data format☆323Updated last month
- Tracking, Benchmarking and Sharing Information about an open source embedded data storage engines, internals, architectures, data storage…☆182Updated 6 years ago
- A SQLite extension for extracting values from serialized Protobuf messages☆88Updated 3 months ago
- FemtoZip is a "shared dictionary" compression library optimized for small documents that may not compress well with traditional tools suc…☆145Updated 4 years ago
- Fast implementation of Content Defined Chunking (CDC) based on a rolling Rabin Checksum in C.☆54Updated 10 years ago
- WebR2sync+☆12Updated 8 years ago
- A minimal C library for operational transform☆111Updated 10 years ago
- A compact implementation of Dr. Askitis HatTrie☆80Updated 11 years ago
- Library implementing the storage and the query evaluation for a text search engine. It uses on a key value store database interface to st…☆47Updated 3 years ago
- UMASH: a fast enough hash and fingerprint with collision bounds☆158Updated 3 weeks ago
- Automatically exported from code.google.com/p/idzip☆40Updated 10 years ago
- Direct IO helpers for block devices and regular files on FreeBSD, Linux, macOS and Windows.☆74Updated 2 years ago
- Benign remote code execution☆112Updated 2 months ago
- Compression and Encryption Virtual File System for SQLite 3.☆118Updated 4 years ago
- BangDB - nosql database☆56Updated 3 weeks ago
- The dream accurate approximate set cardinality estimator based on 3-bit HyperLogLog. More accurate than Redis HyperLogLog.☆55Updated 4 years ago
- immutable sorted string table library☆81Updated 8 months ago
- A CPace PAKE implementation using libsodium.☆36Updated 4 years ago
- SQLite3 extension for read/write storage compression with Zstandard☆185Updated last year
- fast SIMD-able JIT regular expression compiler☆197Updated 10 years ago
- Append-only key-value database on a distributed shared-log☆52Updated last year
- Fast Hash Functions Using AES Intrinsics☆83Updated 6 years ago
- Backtrace.io's resilient log record framing format☆40Updated 3 years ago
- REST API for SQLite☆84Updated 2 years ago
- Fast in-memory database for node and browser☆27Updated last year
- MetroHash v1: Exceptionally fast and statistically robust hash functions☆369Updated 3 years ago
- A space efficient alternative to base-64☆244Updated last year