Proof of concept: Exploiting temporal coherence in LLM inference-- delta encoding for KV cache compression and weight-skip prediction. Achieves F16-quality KV cache at Q4_0 compression ratios with zero perplexity loss on llama.cpp.
☆50Apr 10, 2026Updated 4 months ago
Alternatives and similar repositories for delta-compress-llm
Users that are interested in delta-compress-llm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Editor Code Assistant (ECA) integration for Vscode☆16Jul 28, 2026Updated 3 weeks ago
- A sleek, customizable interface for managing LLMs with responsive design and easy agent personalization.☆19Aug 30, 2024Updated last year
- Specialized AI agents for your bare metal. 100% On-Prem & Air-Gap ready.☆26Feb 21, 2026Updated 5 months ago
- Inference Llama 2 with a model compiled to native code by TorchInductor☆14Feb 8, 2024Updated 2 years ago
- Debian packaging for NNCP [archived], moved to https://salsa.debian.org/go-team/packages/nncp☆14Feb 18, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆18Jul 1, 2025Updated last year
- Open Source Routing Machine for OpenStreetMap API Lib and App for Nim☆10Jun 6, 2019Updated 7 years ago
- Handle Android "draw over other apps" permissions and queries in a version-agnostic way☆22Jun 17, 2019Updated 7 years ago
- It's ooey and gooey. No clue what this really is aside from a GUI framework basis?☆14Oct 1, 2024Updated last year
- ISAAC PRNG implementation on Nim☆10Jul 15, 2026Updated last month
- Build LLM systems you actually control. A free, open engineering book + course — from tokenization to serving your own models. Mechanisms…☆45Aug 9, 2026Updated last week
- A block based Conflict-free Replicated Data Type☆20Apr 23, 2024Updated 2 years ago
- ☆20Sep 28, 2024Updated last year
- A high performance authentication and access-control gateway for LLM API backends☆16Jun 11, 2026Updated 2 months ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Awaitable threadpool for nim☆53May 16, 2025Updated last year
- High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.☆465Feb 22, 2026Updated 5 months ago
- Offline-first, desktop AI assistant tailored for educators, enabling them to generate questions directly from source materials.☆24Aug 2, 2025Updated last year
- ☆24Jan 22, 2025Updated last year
- ☆23Aug 9, 2024Updated 2 years ago
- Using embeddings compressed by Product Quantization, in Javascript☆31Jul 6, 2023Updated 3 years ago
- Rust port of the streaming ddelta patch algorithm, based on bsdiff☆12May 2, 2024Updated 2 years ago
- LLM inference in C/C++☆23Oct 4, 2024Updated last year
- A common data structures for Inter-Blockchain Communication (IBC) messages that can be reused by different implementations or tools☆15Apr 9, 2025Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- native Rust implementation of Kafka protocol and api☆14Jun 13, 2023Updated 3 years ago
- Synchronises Ableton Live to a live input such as microphone in the room or DJ playing record☆11Aug 14, 2014Updated 12 years ago
- VellumForge2 is a Golang CLI for generating high-quality Direct Preference Optimization datasets via a hierarchical prompt pipeline with …☆21Feb 15, 2026Updated 6 months ago
- buck2 oci rules☆12Jul 14, 2026Updated last month
- ⚡ Lightning-fast local log search - grep for logs, but actually good☆27Mar 6, 2026Updated 5 months ago
- A collection of very basic allocators. Written with WebAssembly in mind.☆15Jan 12, 2024Updated 2 years ago
- Additional 3rd party software used in Playbit. See playbit/public for most 3rd party software☆13Jul 14, 2025Updated last year
- Count Min Sketch implementation for Rust☆15Jul 31, 2026Updated 2 weeks ago
- Static Huffman coding☆10Apr 3, 2017Updated 9 years ago
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Nothing to see here yet ...☆13Jun 28, 2025Updated last year
- ☆18Mar 12, 2026Updated 5 months ago
- Compiler for the Lavish IDL☆17Jul 11, 2019Updated 7 years ago
- Universe for growing software.☆49Updated this week
- The H3 Compressor: A compression scheme tailored for H3 cell indexes.☆16Mar 25, 2024Updated 2 years ago
- Model for provider-neutral financial data, with implementation for IEX☆14Jul 30, 2019Updated 7 years ago
- Use a Structure of Arrays like an Array of Structures☆18Nov 7, 2021Updated 4 years ago