Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
☆362May 21, 2025Updated 10 months ago
Alternatives and similar repositories for KVSplit
Users that are interested in KVSplit are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Merliot Device Hub☆166Jun 11, 2025Updated 9 months ago
- Fully neural approach for text chunking☆407Oct 23, 2025Updated 5 months ago
- Artificial Neural Engine Machine Learning Library☆1,523Mar 10, 2026Updated 2 weeks ago
- Docker-based inference engine for AMD GPUs☆233Oct 7, 2024Updated last year
- A simple alternative to homebrew for installing binary packages on MacOS & Linux written in Go.☆217Feb 16, 2026Updated last month
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- 🕵️♂️ All-in-one OSINT tool for analysing any website☆26Nov 11, 2025Updated 4 months ago
- A browser-based, WebGL2 implementation of GPT-2 with transform block and attention matrix visualization☆343Oct 24, 2025Updated 5 months ago
- ☆200May 5, 2025Updated 10 months ago
- Erlang interpreter for Node-RED (visual flow based programming) with Elixir support☆333Updated this week
- A powerful document AI question-answering tool that connects to your local Ollama models. Create, manage, and interact with RAG systems f…☆1,096Aug 9, 2025Updated 7 months ago
- High-Performance Implementation of OpenAI's TikToken.☆473Jul 3, 2025Updated 8 months ago
- Multi-modal OCR pipeline optimized for ML training (text, figure, math, tables, diagrams)☆685May 20, 2025Updated 10 months ago
- Pytorch script hot swap: Change code without unloading your LLM from VRAM☆125Apr 21, 2025Updated 11 months ago
- Official Rust Implementation of Model2Vec☆164Updated this week
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Min.js Style Compression of Tech Docs for LLM Context☆674Oct 5, 2025Updated 5 months ago
- Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full…☆13,197Mar 20, 2026Updated last week
- Rewriting Principia Mathematica in Lean☆137Feb 5, 2026Updated last month
- Browser-LLM Auto-Scaling Technology☆776Jan 29, 2026Updated last month
- Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.☆628Feb 24, 2025Updated last year
- High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and in…☆1,259Mar 20, 2026Updated last week
- LLM plugin for pulling content from Hacker News☆126May 5, 2025Updated 10 months ago
- Browser LLM Demo (like ChatGPT). Works locally with JavaScript and WebGPU☆69Aug 3, 2025Updated 7 months ago
- Resource (icon) extraction tools☆13Apr 22, 2024Updated last year
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Server for Matching Long/Lat to Timezone☆47Feb 21, 2026Updated last month
- Proof of concept for a VPN over UDP☆115Feb 3, 2026Updated last month
- Transductive regular expressions☆254Sep 25, 2025Updated 6 months ago
- A new wide-spectrum content blocker for Safari.☆363Feb 16, 2026Updated last month
- A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your …☆3,195Mar 12, 2026Updated 2 weeks ago
- ☆1,301Aug 21, 2025Updated 7 months ago
- Render source code in 3D, for macOS and iOS.☆197Dec 1, 2024Updated last year
- Real-time guardrail that shows token spend & kills runaway LLM/agent loops.☆157Jul 31, 2025Updated 7 months ago
- A cache for AI agents to learn and replay complex behaviors.☆758Jun 15, 2025Updated 9 months ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click and start building anything your business needs.
- A reimplementation of Stable Diffusion 3.5 in pure PyTorch☆699Jun 14, 2025Updated 9 months ago
- jpg Acropalypse POC☆30Mar 18, 2023Updated 3 years ago
- MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model.☆3,118Jul 7, 2025Updated 8 months ago
- a small, lightweight crate for numerical integration written in Rust.☆114Updated this week
- a concurrent hash array mapped trie implementation in go☆59Jun 19, 2025Updated 9 months ago
- A course of learning LLM inference serving on Apple Silicon for systems engineers: build a tiny vLLM + Qwen.☆4,029Updated this week
- You should own and be able to do anything with YOUR social data, not just the apps, ais, and algoritms of the profit-oriented companies t…☆50Mar 10, 2026Updated 2 weeks ago