Proof of concept: Exploiting temporal coherence in LLM inference-- delta encoding for KV cache compression and weight-skip prediction. Achieves F16-quality KV cache at Q4_0 compression ratios with zero perplexity loss on llama.cpp.
☆50Apr 10, 2026Updated 3 months ago
Alternatives and similar repositories for delta-compress-llm
Users that are interested in delta-compress-llm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Turbo Lossless - 1.33x Smaller, 2.93x Faster, Decode with 1 ADD operation☆15Apr 4, 2026Updated 3 months ago
- A sleek, customizable interface for managing LLMs with responsive design and easy agent personalization.☆19Aug 30, 2024Updated last year
- Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into VRAM☆17May 6, 2026Updated 2 months ago
- Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and in…☆18Nov 11, 2024Updated last year
- Inference Llama 2 with a model compiled to native code by TorchInductor☆14Feb 8, 2024Updated 2 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Handle Android "draw over other apps" permissions and queries in a version-agnostic way☆22Jun 17, 2019Updated 7 years ago
- Random AI notes for working with local models or playing around with random machine learning bits.☆61Jun 7, 2026Updated last month
- It's ooey and gooey. No clue what this really is aside from a GUI framework basis?☆14Oct 1, 2024Updated last year
- ISAAC PRNG implementation on Nim☆10Jul 15, 2026Updated 2 weeks ago
- A block based Conflict-free Replicated Data Type☆20Apr 23, 2024Updated 2 years ago
- Feedforward Post Compression for Dynamic Gaussian Splatting (for standard GS sequence)☆21Dec 23, 2025Updated 7 months ago
- ☆12Jun 30, 2022Updated 4 years ago
- ☆20Sep 28, 2024Updated last year
- High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.☆464Feb 22, 2026Updated 5 months ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- Play any game ROM with associated emulator in RetroArch on Linux☆18Jan 10, 2025Updated last year
- Asynchronous downloads manager library for Nim.☆12Jan 12, 2024Updated 2 years ago
- This crate includes a low level implementation of the PMTiles format based on the standard Read and Write trait.☆18Mar 31, 2026Updated 3 months ago
- Offline-first, desktop AI assistant tailored for educators, enabling them to generate questions directly from source materials.☆24Aug 2, 2025Updated 11 months ago
- ☆24Jan 22, 2025Updated last year
- A fully offline voice assistant that combines lmstudio and applio together. Uses two methods of TTS, STT and also has some extra features…☆23Apr 14, 2025Updated last year
- A small PHP program to send SMS messages to a GOIP GSM Gateway using a simple restful API☆12Dec 28, 2024Updated last year
- ☆23Aug 9, 2024Updated last year
- Rust port of the streaming ddelta patch algorithm, based on bsdiff☆12May 2, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- LLM inference in C/C++☆23Oct 4, 2024Updated last year
- A common data structures for Inter-Blockchain Communication (IBC) messages that can be reused by different implementations or tools☆15Apr 9, 2025Updated last year
- terminal ui widget based on illwill☆20May 26, 2026Updated 2 months ago
- Tiny file dialogs bindings for Nim☆21Oct 29, 2025Updated 9 months ago
- A parser for PTX 6.5☆13Jun 19, 2023Updated 3 years ago
- VellumForge2 is a Golang CLI for generating high-quality Direct Preference Optimization datasets via a hierarchical prompt pipeline with …☆20Feb 15, 2026Updated 5 months ago
- ⚡ Lightning-fast local log search - grep for logs, but actually good☆27Mar 6, 2026Updated 4 months ago
- Official code of "ADC-GS: Anchor-Driven Deformable and Compressed Gaussian Splatting for Dynamic Scene Reconstruction", IJCAI2025☆32Mar 10, 2026Updated 4 months ago
- the most complete serialization tree for serde☆16Apr 24, 2024Updated 2 years ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Compiler for the Lavish IDL☆17Jul 11, 2019Updated 7 years ago
- ☆12Dec 3, 2025Updated 7 months ago
- Classifies percussion audio samples with a CNN-LSTM, written in python and pytorch. Also exports to Drumkv1 (lv2 plugin)☆14Aug 20, 2020Updated 5 years ago
- Model for provider-neutral financial data, with implementation for IEX☆14Jul 30, 2019Updated 6 years ago
- Yet another frontend for LLM, written using .NET and WinUI 3☆11Sep 14, 2025Updated 10 months ago
- A collection of nature-inspired metaheuristic algorithms.☆16May 2, 2024Updated 2 years ago
- ☆27Sep 25, 2024Updated last year