Ying1123 / llm-caching-multiplexingLinks
☆20Updated 2 years ago
Alternatives and similar repositories for llm-caching-multiplexing
Users that are interested in llm-caching-multiplexing are comparing it to the libraries listed below
Sorting:
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆68Updated 10 months ago
- Code associated with the paper **Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees**.☆27Updated 2 years ago
- ☆19Updated 2 years ago
- [ICDCS 2023] Evaluation and Optimization of Gradient Compression for Distributed Deep Learning☆10Updated 2 years ago
- Compression for Foundation Models☆35Updated 2 months ago
- ☆26Updated 2 years ago
- Official resporitory for "IPDPS' 24 QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices".☆20Updated last year
- Elixir: Train a Large Language Model on a Small GPU Cluster☆15Updated 2 years ago
- An Attention Superoptimizer☆22Updated 8 months ago
- Repository for the COLM 2025 paper SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths☆12Updated 3 months ago
- Artifact for "Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving" [SOSP '24]☆25Updated 10 months ago
- ☆32Updated 7 months ago
- [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention☆47Updated 2 months ago
- ☆14Updated 2 months ago
- ☆93Updated 3 years ago
- ☆22Updated 4 years ago
- 🔥 LLM-powered GPU kernel synthesis: Train models to convert PyTorch ops into optimized Triton kernels via SFT+RL. Multi-turn compilation…☆79Updated last week
- [ICDCS 2023] DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining☆11Updated last year
- ☆47Updated last year
- APRIL: Active Partial Rollouts in Reinforcement Learning to Tame Long-tail Generation. A system-level optimization for scalable LLM tra…☆25Updated 2 weeks ago
- PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".☆90Updated 2 years ago
- Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference☆46Updated last year
- How much energy do GenAI models consume?☆47Updated 4 months ago
- Beyond KV Caching: Shared Attention for Efficient LLMs☆19Updated last year
- Official Repo for "SplitQuant / LLM-PQ: Resource-Efficient LLM Offline Serving on Heterogeneous GPUs via Phase-Aware Model Partition and …☆34Updated last month
- ☆38Updated last year
- Odysseus: Playground of LLM Sequence Parallelism☆77Updated last year
- A resilient distributed training framework☆95Updated last year
- GPU operators for sparse tensor operations☆35Updated last year
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆41Updated 2 years ago