Lyn-Lucy / MSDLinks
☆28Updated 3 months ago
Alternatives and similar repositories for MSD
Users that are interested in MSD are comparing it to the libraries listed below
Sorting:
- 📰 Must-read papers on KV Cache Compression (constantly updating 🤗).☆559Updated 3 weeks ago
- Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.☆376Updated 7 months ago
- Curated collection of papers in MoE model inference☆285Updated last month
- 📰 Must-read papers and blogs on Speculative Decoding ⚡️☆979Updated last month
- This repository serves as a comprehensive survey of LLM development, featuring numerous research papers along with their corresponding co…☆221Updated 2 months ago
- Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline mod…☆562Updated last year
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)☆320Updated 5 months ago
- Summary of some awesome work for optimizing LLM inference☆120Updated 4 months ago
- Curated collection of papers in machine learning systems☆426Updated 2 weeks ago
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)☆155Updated last year
- ☆29Updated 7 months ago
- The repository has collected a batch of noteworthy MLSys bloggers (Algorithms/Systems)☆285Updated 9 months ago
- ☆24Updated last year
- Awesome list for LLM quantization☆318Updated last week
- Paper reading and discussion notes, covering AI frameworks, distributed systems, cluster management, etc.☆23Updated 7 months ago
- All Homeworks for TinyML and Efficient Deep Learning Computing 6.5940 • Fall • 2023 • https://efficientml.ai☆182Updated last year
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆278Updated 4 months ago
- Puzzles for learning Triton, play it with minimal environment configuration!☆545Updated 3 weeks ago
- 📚200+ Tensor/CUDA Cores Kernels, ⚡️flash-attn-mma, ⚡️hgemm with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS/FA2 🎉🎉).☆46Updated 5 months ago
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆278Updated 7 months ago
- This repository is established to store personal notes and annotated papers during daily research.☆155Updated 2 weeks ago
- Disaggregated serving system for Large Language Models (LLMs).☆706Updated 6 months ago
- Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)☆64Updated 5 months ago
- Train speculative decoding models effortlessly and port them smoothly to SGLang serving.☆428Updated last week
- ☆138Updated 4 months ago
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆338Updated 3 months ago
- Galvatron is an automatic distributed training system designed for Transformer models, including Large Language Models (LLMs). If you hav…☆23Updated 7 months ago
- Fast inference from large lauguage models via speculative decoding☆835Updated last year
- paper and its code for AI System☆331Updated 2 months ago
- ☆78Updated 3 weeks ago