cornserve-ai / cornserveLinks
Easy, Fast, and Scalable Multimodal AI
☆83Updated last week
Alternatives and similar repositories for cornserve
Users that are interested in cornserve are comparing it to the libraries listed below
Sorting:
- [ICLR2025] Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding☆135Updated last year
- KV cache compression for high-throughput LLM inference☆148Updated 11 months ago
- Efficient Long-context Language Model Training by Core Attention Disaggregation☆73Updated last week
- ☆48Updated last year
- LLM Serving Performance Evaluation Harness☆82Updated 10 months ago
- 🔥 LLM-powered GPU kernel synthesis: Train models to convert PyTorch ops into optimized Triton kernels via SFT+RL. Multi-turn compilation…☆110Updated last month
- [NeurIPS 2025] Scaling Speculative Decoding with Lookahead Reasoning☆58Updated 2 months ago
- Distributed MoE in a Single Kernel [NeurIPS '25]☆174Updated this week
- GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM☆175Updated last year
- [ICLR'25] Fast Inference of MoE Models with CPU-GPU Orchestration☆253Updated last year
- An early research stage expert-parallel load balancer for MoE models based on linear programming.☆484Updated last month
- A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM☆182Updated this week
- ☆116Updated 7 months ago
- 16-fold memory access reduction with nearly no loss☆109Updated 9 months ago
- torchcomms: a modern PyTorch communications API☆319Updated this week
- Block Diffusion for Ultra-Fast Speculative Decoding☆188Updated this week
- [NeurIPS 2025] Simple extension on vLLM to help you speed up reasoning model without training.☆216Updated 7 months ago
- ☆81Updated 2 months ago
- [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank☆66Updated last year
- Code for data-aware compression of DeepSeek models☆66Updated 3 weeks ago
- Vortex: A Flexible and Efficient Sparse Attention Framework☆43Updated last month
- [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable☆204Updated last year
- Efficient LLM Inference over Long Sequences☆394Updated 6 months ago
- ☆63Updated 7 months ago
- The evaluation framework for training-free sparse attention in LLMs☆108Updated 2 months ago
- Bamboo-7B Large Language Model☆93Updated last year
- ByteCheckpoint: An Unified Checkpointing Library for LFMs☆256Updated last month
- Artifact for "Marconi: Prefix Caching for the Era of Hybrid LLMs" [MLSys '25 Outstanding Paper Award, Honorable Mention]☆47Updated 10 months ago
- Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference☆160Updated 2 months ago
- ☆144Updated last year