Proof of concept: Exploiting temporal coherence in LLM inference-- delta encoding for KV cache compression and weight-skip prediction. Achieves F16-quality KV cache at Q4_0 compression ratios with zero perplexity loss on llama.cpp.
☆51Apr 10, 2026Updated 6 months ago
Alternatives and similar repositories for delta-compress-llm
Users that are interested in delta-compress-llm are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Turbo Lossless - 1.33x Smaller, 2.93x Faster, Decode with 1 ADD operation☆16Apr 4, 2026Updated 7 months ago
- ☆13Dec 20, 2017Updated 8 years ago
- Fast, lightweight microVM for cloud streaming that runs on a shared GPU at baremetal speed☆50Oct 16, 2026Updated 3 weeks ago
- Verified multi-source distribution for local LLM weights☆17Updated this week
- A sleek, customizable interface for managing LLMs with responsive design and easy agent personalization.☆19Aug 30, 2024Updated 2 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Lab 5 project of MIT-6.5940, deploying LLaMA2-7B-chat on one's laptop with TinyChatEngine.☆17Dec 1, 2023Updated 2 years ago
- Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into VRAM☆17May 6, 2026Updated 6 months ago
- Specialized AI agents for your bare metal. 100% On-Prem & Air-Gap ready.☆25Feb 21, 2026Updated 8 months ago
- ☆18Jul 1, 2025Updated last year
- 🌍 LLOT - Local LLM Ollama Translator | Privacy-first translation with 40+ languages | Self-hosted☆14Updated this week
- Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and in…☆18Nov 11, 2024Updated last year
- Inference Llama 2 with a model compiled to native code by TorchInductor☆14Feb 8, 2024Updated 2 years ago
- High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.☆469Feb 22, 2026Updated 8 months ago
- 🍎👉🍏 Everything you need in order to get started building distributed machine learning with Apple's MLX☆23Aug 9, 2025Updated last year
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Google Pixel 6 custom kernel based on android14-6.1 branch☆14Oct 25, 2026Updated last week
- Claude Code execution playbook with 3 pilot modes: cost-first (Haiku), quality-first (Sonnet), ceiling-elevation (Opus). Quantitative esc…☆24May 8, 2026Updated 6 months ago
- Advanced image processing nodes for ComfyUI with DeepInv, BM3D, wavelet denoising, and more☆12Dec 20, 2025Updated 10 months ago
- A block based Conflict-free Replicated Data Type☆20Apr 23, 2024Updated 2 years ago
- [ICML 2025] CommVQ: Commutative Vector Quantization for KV Cache Compression☆27Sep 2, 2025Updated last year
- Stream movies together with your friends. Host your media library on your Android or Linux device.☆14Nov 7, 2021Updated 5 years ago
- A set of ZBrush plugins☆15Jun 28, 2019Updated 7 years ago
- Offline-first, desktop AI assistant tailored for educators, enabling them to generate questions directly from source materials.☆24Aug 2, 2025Updated last year
- Creating AI agents with the simplicity of Python in Java.☆14Mar 21, 2026Updated 7 months ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆20Sep 28, 2024Updated 2 years ago
- Reliable transcription, with captions cut for every screen shape from one file. An open caption format, a video.js plugin with style pack…☆58Updated this week
- torchlogic is a pytorch framework for developing Neuro-Symbolic AI systems and implements Neural Reasoning Networks.☆23Oct 24, 2026Updated 2 weeks ago
- ☆23Aug 9, 2024Updated 2 years ago
- cnccoder is a crate for writing cutting instructions and converting them to G-code for use on 3 axis CNC machines☆11Apr 29, 2024Updated 2 years ago
- ☆24Jan 22, 2025Updated last year
- ☆11Apr 17, 2023Updated 3 years ago
- Support library for using ImGui in your CLAP plugin☆11Aug 18, 2026Updated 2 months ago
- native Rust implementation of Kafka protocol and api☆14Jun 13, 2023Updated 3 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- 模型加速/模型压缩（已完成所有Lab）☆11Dec 24, 2023Updated 2 years ago
- A SpinBox widget prototype for Textual☆11Aug 10, 2025Updated last year
- ☆13Jun 30, 2022Updated 4 years ago
- VS Code extension to manage Taskwarrior tasks☆10Jul 19, 2023Updated 3 years ago
- Node.js proxy server for debugging OpenAI-compatible chat completions requests.☆10Nov 20, 2025Updated 11 months ago
- Wiki for design work going beyond Rust's current reference types☆14Oct 2, 2026Updated last month
- LLM inference in C/C++☆23Oct 4, 2024Updated 2 years ago