Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
☆361May 21, 2025Updated last year
Alternatives and similar repositories for KVSplit
Users that are interested in KVSplit are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Fully neural approach for text chunking☆420Oct 23, 2025Updated last year
- Merliot Device Hub☆162Jun 11, 2025Updated last year
- Artificial Neural Engine Machine Learning Library☆1,703Updated this week
- High-Performance Implementation of OpenAI's TikToken.☆474Jul 3, 2025Updated last year
- Docker-based inference engine for AMD GPUs☆233Oct 7, 2024Updated 2 years ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Create captivating video tours of your data, ready for sharing on platforms like YouTube, Instagram and TikTok.☆13Jul 3, 2025Updated last year
- Erlang interpreter for Node-RED (visual flow based programming) with Elixir support☆333Updated this week
- A powerful document AI question-answering tool that connects to your local Ollama models. Create, manage, and interact with RAG systems f…☆1,096Aug 9, 2025Updated last year
- Min.js Style Compression of Tech Docs for LLM Context☆670Oct 5, 2025Updated last year
- Pytorch script hot swap: Change code without unloading your LLM from VRAM☆127Apr 21, 2025Updated last year
- A cache for AI agents to learn and replay complex behaviors.☆765Jun 15, 2025Updated last year
- ☆196May 5, 2025Updated last year
- Achieve the llama3 inference step-by-step, grasp the core concepts, master the process derivation, implement the code.☆630Feb 24, 2025Updated last year
- A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your …☆3,321Sep 30, 2026Updated last month
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Open-source VMs-as-a-service☆807Aug 30, 2026Updated 2 months ago
- I made my AI think harder by making it argue with itself repeatedly. It works stupidly well.☆2,326May 2, 2025Updated last year
- This repo contains a new way to use bloom filters to do lossless video compression☆249Jun 5, 2025Updated last year
- https://ollama.com/search?o=newest☆37Updated this week
- AI Dataset Generator – Create realistic datasets for demos, learning, and dashboards☆765Oct 19, 2026Updated 2 weeks ago
- ☆1,317Aug 21, 2025Updated last year
- A new wide-spectrum content blocker for Safari.☆355Jun 26, 2026Updated 4 months ago
- High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and in…☆1,315Updated this week
- Multi-modal OCR pipeline optimized for ML training (text, figure, math, tables, diagrams)☆674May 13, 2026Updated 5 months ago
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Resource (icon) extraction tools☆13Apr 22, 2024Updated 2 years ago
- browser plugin to send youtube, insta (all social videos) to local backend and process audio and video in all sorts of ways.☆250Aug 31, 2025Updated last year
- A browser-based, WebGL2 implementation of GPT-2 with transform block and attention matrix visualization☆348Oct 24, 2025Updated last year
- LLM plugin for pulling content from Hacker News☆127May 5, 2025Updated last year
- ☆11Apr 1, 2025Updated last year
- A structural code search engine for Al agents.☆553Jul 13, 2026Updated 3 months ago
- The open-source managed agent platform. A self-hostable alternative to Claude Managed Agents and OpenAI's Agents API.☆3,342Updated this week
- macOS multi-mouse & multi-external display assignment for complex screen and peripheral workstations.☆278May 27, 2025Updated last year
- Run models too big for your Mac's memory☆669Oct 3, 2026Updated last month
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- MiniMax-M1, the world's first open-weight, large-scale hybrid-attention reasoning model.☆3,186Jul 7, 2025Updated last year
- TFT (Training from Traces) benchmark: 5 scenarios comparing TFT pipeline vs Direct Training on SGD restaurant traces☆13Sep 7, 2026Updated last month
- Offline SOS signaling and recovery app for wars and disasters (iOS & Android) — like a digital flare-gun.☆343May 2, 2025Updated last year
- 🧠 Retrieval Augmented Generation (RAG) example☆20Apr 17, 2026Updated 6 months ago
- RAG Logger is an open-source logging tool designed specifically for Retrieval-Augmented Generation (RAG) applications. It serves as a lig…☆226Dec 24, 2024Updated last year
- A reimplementation of Stable Diffusion 3.5 in pure PyTorch☆706Jun 14, 2025Updated last year
- Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE)☆2,750Sep 11, 2026Updated last month