finnchen11 / VLLM_PromptCacheLinks
Optimize vLLM with persistent system prompt caching and block reuse for faster, memory-efficient inference.
☆53Updated last month
Alternatives and similar repositories for VLLM_PromptCache
Users that are interested in VLLM_PromptCache are comparing it to the libraries listed below
Sorting:
- [NeurIPS 2025] Official implementation of "STRAP: Spatio-Temporal Pattern Retrieval for Out-of-Distribution Generalization"☆72Updated 3 weeks ago
- ☆79Updated this week
- Kubernetes Operator for managing OpenResty with custom CRDs (OpenResty, Server, Location, Upstream, RateLimitPolicy)☆50Updated 5 months ago
- A pytorch implementation of the paper "TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Simi…☆343Updated last month
- Cascade is a production-ready, high-performance, and low-latency audio stream processing library designed for Voice Activity Detection (V…☆82Updated this week
- [DIVER] Breaking Imitation Bottlenecks: Reinforced Diffusion Powers Diverse Trajectory Generation☆101Updated 3 weeks ago
- ☆42Updated 7 months ago
- Dataset and evaluation code of ISDrama(ACM-MM 2025): Immersive Spatial Drama Generation through Multimodal Prompting☆234Updated 3 months ago
- Desktop Tiny Agent is a lightweight, modular desktop intelligent agent framework. It offers plugin extensibility, task scheduling (sync/a…☆80Updated 2 months ago
- ☆43Updated 3 months ago
- ☆138Updated last year
- ☆22Updated last month
- A bibliometric visualization platform that integrates Gestalt design principles, keyword extraction algorithms, temporal algorithms, mach…☆89Updated 3 weeks ago
- ☆42Updated 10 months ago
- a rather fast time struct getter☆80Updated 4 months ago
- ☆41Updated last year
- ☆144Updated last year
- CommercialGoatAPI is a commercial project that provides remote HTTP access to Goat API(and alias API) supporting all interfaces of these …☆83Updated last month
- Source code of Fuyao, built on Nightcore☆17Updated last year
- ☆135Updated last year
- ☆43Updated last week
- Advanced Driving Assistance System based on Jetson Nano☆84Updated 4 months ago
- Main Project of AIDE☆91Updated 9 months ago
- 一键生成专业级播客 - AI 播客工作流服务☆41Updated 2 weeks ago
- Kafka Dog is a lightweight desktop application for visualizing and managing Apache Kafka. It provides a user-friendly graphical interface…☆126Updated 11 months ago
- A lightweight and easy-to-use RPC framework created by Bruce Pang☆124Updated 9 months ago
- Example project using universal links as deeplinks to switch iOS apps.☆13Updated last year
- Analysis and visualization of multi-omics data. In ongoing development: multi-modal fusion, sparse learning, and spatio-temporal effects.…☆207Updated 8 months ago
- 用Hexo的方式管理TypeCho(使用Github Actions自动更新文章到TypeCho)☆83Updated 7 months ago
- kight is a static analysis tool for c/c++ programs.☆213Updated 10 months ago