Smart OpenAI‑compatible proxy for llama.cpp: manages slots, saves/restores KV cache to disk, routes requests by prefix similarity, and protects hot slots from being overwritten. Accelerates long prompts (30–60k tokens) via instant reuse or fast on‑demand restore; supports SSE streaming and non‑stream JSON over /v1/chat/completions.
☆34Nov 14, 2025Updated 3 months ago
Alternatives and similar repositories for proxycache
Users that are interested in proxycache are comparing it to the libraries listed below
Sorting:
- AdaLLM is an NVFP4-first inference runtime for Ada Lovelace (RTX 4090) with FP8 KV cache and custom decode kernels. This repo targets NVF…☆96Feb 15, 2026Updated 2 weeks ago
- Listen to your favorite internet radio stations on GNOME!☆26Dec 26, 2025Updated 2 months ago
- this is to send custom emails from google spreadsheet using google app scripts☆15Nov 30, 2024Updated last year
- Retrieve and analysis data from SDSS (Sloan Digital Sky Survey)☆10Nov 1, 2023Updated 2 years ago
- MCP tools for Rust Context Engineering (rustdocs, rust analyzer)☆14Feb 8, 2026Updated 3 weeks ago
- Dynamic Swagger UI for frappe Apps☆19Sep 30, 2024Updated last year
- MCP (Model Context Protocol) server for Listmonk newsletter management☆24Jan 13, 2026Updated last month
- Helper scripts to be run on Frappe sites☆16Dec 19, 2025Updated 2 months ago
- Push Notification Relay Server for Frappe Apps☆12Dec 24, 2025Updated 2 months ago
- Raspberry Pi based automated garden irrigation system☆10Mar 4, 2023Updated 3 years ago
- A simple Mojolicious application example for authenticating a user and maintaining a session☆11Sep 20, 2019Updated 6 years ago
- To maintain ICAO (3-letter) airline names mapping to their full name☆12Jan 10, 2026Updated last month
- Proxy for OpenAI☆15Sep 2, 2025Updated 6 months ago
- ☆15Apr 3, 2025Updated 11 months ago
- Mojolicious lite and bootstrap based simple cms☆10Nov 25, 2022Updated 3 years ago
- Прокси-сервер для подключения Алисы к Dialogflow☆13Mar 23, 2021Updated 4 years ago
- Visual Tagger is a JavaScript tool that visually highlights HTML elements for AIs, aiding in identifying interactive components on web pa…☆11Oct 28, 2024Updated last year
- Scripts and tools for optimizing quantizations in llama.cpp with GGUF imatrices.☆18Jan 10, 2025Updated last year
- ☆16Jun 16, 2024Updated last year
- Fork of "F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching"☆17Nov 27, 2024Updated last year
- Custom implementation based on Bonet and Wood's Nonlinear FEM book http://www.flagshyp.com☆14Jan 23, 2026Updated last month
- a .net port of GhostCursor https://github.com/Xetera/ghost-cursor☆12May 30, 2021Updated 4 years ago
- Specialized MiniMax Model Context Protocol (MCP) server designed for coding-plan users, featuring AI-powered search and vision analysis A…☆31Feb 10, 2026Updated 3 weeks ago
- You can run passively cooled single slot Tesla GPU/KI cards in a HP Proliant with a modified iLO ROM that take care of the fan and conseq…☆17Dec 16, 2025Updated 2 months ago
- Working with LLM in C#☆15Updated this week
- world's stupidest moe llm in 103M parameters☆20Jul 18, 2025Updated 7 months ago
- ☆11Nov 10, 2024Updated last year
- Checkfront API☆17Feb 27, 2025Updated last year
- Рускоговорящий GLaDOS анти-ассистент☆14Jun 23, 2024Updated last year
- ☆23Dec 8, 2025Updated 2 months ago
- Spectral analysis and training of dense layers☆17Jan 12, 2024Updated 2 years ago
- a frappe app for my medium post☆15Jun 19, 2024Updated last year
- Downloads books from the amazon web reader☆30Oct 15, 2025Updated 4 months ago
- Verify correctness of ddrescue images, using MD5 hashes☆11Dec 30, 2025Updated 2 months ago
- Gives realtime and asynchronous feedback to parts created in the FreeCAD Part Design and Sketcher workbenches.☆16Dec 29, 2024Updated last year
- Simple node proxy for llama-server that enables MCP use☆17May 10, 2025Updated 9 months ago
- Connect Channel Messenger, Zalo, Viber, Skype, Telegram☆13Nov 5, 2018Updated 7 years ago
- NPX/Docker package that creates Ollama API server and forward requests to Gemni/OpenAI/Deepseek/Kimi K2. Mainly purpose to use Free tier …☆29Oct 5, 2025Updated 5 months ago
- IceCash. Касса Linux. Рабочее место кассира под linux с использованием web интерфейса. С драйвером к Штрих-М ФРК.☆25Jun 9, 2015Updated 10 years ago