Smart OpenAI‑compatible proxy for llama.cpp: manages slots, saves/restores KV cache to disk, routes requests by prefix similarity, and protects hot slots from being overwritten. Accelerates long prompts (30–60k tokens) via instant reuse or fast on‑demand restore; supports SSE streaming and non‑stream JSON over /v1/chat/completions.
☆57Sep 16, 2026Updated last month
Alternatives and similar repositories for proxycache
Users that are interested in proxycache are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆27Oct 7, 2026Updated 3 weeks ago
- LLM inference in C/C++☆12Oct 18, 2026Updated 2 weeks ago
- Make Qwen3 Think like Gemini 2.5 Pro | Open webui function☆24May 10, 2025Updated last year
- LLM backed Fantasy Tribe Game☆19Nov 21, 2024Updated last year
- ☆29Oct 11, 2026Updated 3 weeks ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- llama.cpp with Mirai S (2.4-bit Qwen3.8-27B) support: 128K context on a 12 GB GPU☆11Oct 7, 2026Updated 3 weeks ago
- Development enviroment in docker.☆17Jun 21, 2016Updated 10 years ago
- AI Native Language + Compiler. All LLM☆27Updated this week
- Fast LLM inference in pure C++☆19Jul 18, 2026Updated 3 months ago
- Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.☆161Updated this week
- A fork of https://github.com/Niko1221/Strata optimized for my 3090+5070 Ti setup, see docs/COMPARISON.md. Renamed to Lamina.☆58Updated this week
- pi.dev + llama.cpp ❤️☆27Oct 15, 2026Updated 3 weeks ago
- Generate a llama-quantize command to copy the quantization parameters of any GGUF☆36Apr 20, 2026Updated 6 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆22Oct 21, 2026Updated 2 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- 💜 The slightly more compromising Python code formatter☆11Feb 26, 2021Updated 5 years ago
- AI Task Delegation, subsidize the cloud costs.☆21Jul 8, 2026Updated 3 months ago
- benchmarks for LLM tokenizers☆22Mar 25, 2026Updated 7 months ago
- Turbo Lossless - 1.33x Smaller, 2.93x Faster, Decode with 1 ADD operation☆16Apr 4, 2026Updated 7 months ago
- Private-first, self-hostable knowledge base. Your data, your server, your control. No cloud, no telemetry, no trust required.☆22Oct 4, 2026Updated last month
- Receive.Link: receive files only you can open.☆14Sep 7, 2026Updated 2 months ago
- Agentic BYOK Browser-Based Website Builder☆66Updated this week
- An example repo full of Douglas Adams quotes☆16Mar 12, 2024Updated 2 years ago
- A pi coding-agent terminal running entirely in the browser with Ghostty Web and Pyodide☆13Oct 22, 2026Updated 2 weeks ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- LLM (v)RAM estimator for GGUF models from huggingface and ollama across various quantisation and context sizes☆13Nov 26, 2025Updated 11 months ago
- ☆21Aug 17, 2026Updated 2 months ago
- Scripts to setup a two-node llama.cpp cluster on NVIDIA DGX Spark (GB10) hardware.☆12May 10, 2026Updated 5 months ago
- Transforms raw text into interconnected knowledge graphs and autonomously generates insights through advanced pattern detection. It combi…☆11Jul 29, 2026Updated 3 months ago
- ☆16Updated this week
- LLM inference in C/C++☆35Oct 15, 2026Updated 3 weeks ago
- High-performance batched Top-K selection for CPU inference. Up to 80x faster than PyTorch, optimized for LLM sampling with AVX2 SIMD.☆18Mar 20, 2026Updated 7 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆3,377Updated this week
- turn qwen 3.6 27B-> AR -> diffusion (opendllm + d3llm)☆15Jun 11, 2026Updated 4 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Reproducible SGLang image for DeepSeek-V4-Flash-0731 on SM120☆38Aug 25, 2026Updated 2 months ago
- Reliable model swapping for any local AI server. Chosen by thousands of users. Batteries included. If we don't have it, you probably don…☆6,180Updated this week
- Yet Another (LLM) Web UI, made with Gemini☆12Dec 25, 2024Updated last year
- nagi-llm is a modern evolution of the NAGI interpreter, enhanced with AI and LLM-driven scripting.☆14Dec 17, 2025Updated 10 months ago
- A high performance authentication and access-control gateway for LLM API backends☆17Jun 11, 2026Updated 4 months ago
- HRM-MoE: Efficient Sparse Pretraining with Hierarchical Reasoning☆11Jun 17, 2026Updated 4 months ago
- An open-source Parsec client for Android. This repo implements mouse & keyboard & touch screen support.☆16May 31, 2026Updated 5 months ago