dipampaul17 / KVSplit
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
☆287Updated this week
Alternatives and similar repositories for KVSplit
Users that are interested in KVSplit are comparing it to the libraries listed below
Sorting:
- ☆279Updated 4 months ago
- ⚙️ Create and run workflows (RPA 2.0)☆492Updated this week
- fractal-structure inspired, parent-children orbiting, zooming-elements based interactive graph visualization user interface☆129Updated 2 months ago
- Docker-based inference engine for AMD GPUs☆230Updated 7 months ago
- ai for jq☆240Updated 7 months ago
- ☆159Updated last month
- ☆191Updated last week
- ☆82Updated 6 months ago
- Examples and guides for using the VLM Run API☆276Updated last week
- Turn your Apple Watch into an ammeter to measure DC currents☆193Updated 8 months ago
- A monitoring station for carnivorous flora.☆118Updated last week
- Run and explore Llama models locally with minimal dependencies on CPU☆189Updated 7 months ago
- Min.js Style Compression of Tech Docs for LLM Context☆443Updated this week
- This project collects GPU benchmarks from various cloud providers and compares them to fixed per token costs. Use our tool for efficient …☆221Updated 5 months ago
- Browser-LLM Auto-Scaling Technology☆507Updated last week
- ☆131Updated 2 weeks ago
- Better Bookmarks Search w/ Transformers☆194Updated last year
- Implement recursion using English as the programming language and an LLM as the runtime.☆233Updated 2 years ago
- Documentation and code for Hack the MontyHome device for extended applications.☆231Updated 5 months ago
- 1fps.video client app☆163Updated 3 months ago
- GUI for selecting text files for concatenation and submission to LLMs☆166Updated last week
- vtc: Video Traffic Counter☆60Updated 5 months ago
- SyncLite : Build Anything Sync Anywhere☆151Updated 6 months ago
- Long-Range Data Bridge (LoRaBridge) project repository☆88Updated 3 weeks ago
- A very simple tool to build LLM prompts from your code repositories.☆153Updated 3 months ago
- Attempt to create an Open Source Privacy Focused Rewind.ai Alternative for data capture☆211Updated 3 months ago
- ☆163Updated 11 months ago
- Multi-model transactional embedded database☆68Updated 5 months ago
- Numscript is a Domain-Specific Language (DSL) designed to help you model complex financial transactions, replacing complex and error-pron…☆87Updated this week
- Finds the school district associated with a given street address in the United States☆48Updated 9 months ago