Ying1123 / llm-caching-multiplexing
☆17Updated last year
Related projects: ⓘ
- ☆19Updated last year
- Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference☆21Updated 3 months ago
- A minimal implementation of vllm.☆29Updated last month
- Official Repo for "LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization"☆25Updated 6 months ago
- Code associated with the paper **Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees**.☆24Updated last year
- ☆35Updated 2 months ago
- AN EFFICIENT AND GENERAL FRAMEWORK FOR LAYERWISE-ADAPTIVE GRADIENT COMPRESSION☆10Updated 10 months ago
- Python package for rematerialization-aware gradient checkpointing☆22Updated 10 months ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆29Updated 6 months ago
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆58Updated this week
- An Attention Superoptimizer☆19Updated 4 months ago
- [ICDCS 2023] DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining☆12Updated 9 months ago
- TensorRT LLM Benchmark Configuration☆10Updated last month
- Stateful LLM Serving☆25Updated last month
- ☆24Updated last year
- Elixir: Train a Large Language Model on a Small GPU Cluster☆12Updated last year
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆33Updated last year
- A Suite for Parallel Inference of Diffusion Transformers (DiTs) on multi-GPU Clusters☆25Updated last month
- ☆30Updated 3 months ago
- Code for Palu: Compressing KV-Cache with Low-Rank Projection☆39Updated this week
- PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".☆70Updated last year
- ☆29Updated 3 weeks ago
- ☆16Updated last year
- [ICLR 2024] Jaiswal, A., Gan, Z., Du, X., Zhang, B., Wang, Z., & Yang, Y. Compressing llms: The truth is rarely pure and never simple.☆15Updated 6 months ago
- Activation-aware Singular Value Decomposition for Compressing Large Language Models☆43Updated 3 weeks ago
- Implementation of Hyena Hierarchy in JAX☆10Updated last year
- Odysseus: Playground of LLM Sequence Parallelism☆50Updated 3 months ago
- Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry☆36Updated 8 months ago
- ☆22Updated 3 years ago
- ☆47Updated 3 weeks ago