dipampaul17 / KVSplitLinks
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
☆350Updated 2 weeks ago
Alternatives and similar repositories for KVSplit
Users that are interested in KVSplit are comparing it to the libraries listed below
Sorting:
- Browser-LLM Auto-Scaling Technology☆520Updated this week
- Attempt to create an Open Source Privacy Focused Rewind.ai Alternative for data capture☆213Updated 4 months ago
- ☆279Updated 5 months ago
- ☆195Updated last month
- A tool for enhancing Claude Code☆389Updated this week
- Fully neural approach for text chunking☆353Updated last month
- Applying the ideas of Deepseek R1 to computer use☆213Updated 4 months ago
- HTTP API for Claude Code, Goose, Aider, and Codex☆555Updated this week
- Docker-based inference engine for AMD GPUs☆230Updated 8 months ago
- ai for jq☆241Updated 8 months ago
- Examples and guides for using the VLM Run API☆278Updated this week
- A hub for various industry-specific schemas to be used with VLMs.☆513Updated last week
- The only fully local production-grade Super SDK that provides a simple, unified, and powerful interface for calling more than 200+ LLMs.☆475Updated 2 weeks ago
- Local coding agent with neat UI☆192Updated 2 weeks ago
- CleverBee - The Open Source Deep Researcher Tool☆295Updated last month
- Minimal AI agent framework that just works with only seven tools.☆507Updated 3 weeks ago
- We put browsers on a unikernel☆319Updated this week
- Your toolkit for autonomous, evolving agent ecosystems. Create, execute, govern, and evolve agents that learn from experience, collaborat…☆432Updated this week
- A lightweight Model Context Protocol (MCP) server that enables AI assistants like Claude to retrieve and interpret real-time weather data…☆222Updated this week
- ☆461Updated last week
- Uses an llm to generate ffmpeg commands☆479Updated 4 months ago
- This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs. Use our tool for efficient …☆221Updated 5 months ago
- Securely run AI-generated code in stateful sandboxes that run forever.☆196Updated last month
- ☆160Updated 2 months ago
- ☆131Updated last month
- Fine-grained control over model context protocol (MCP) clients, servers, and tools. Context is God.☆112Updated last month
- A monitoring station for carnivorous flora.☆125Updated 3 weeks ago
- A GTK graphical interface for chatting with large language models (LLMs)☆80Updated this week
- Simple to install, powerful command-line based AI agent system for coding.☆490Updated 2 months ago
- A lightweight protocol-aware packet analyzer and behavioral exporter. Created as a personal response to global academic freedom challenge…☆236Updated this week