keli-wen / AGI-Study
The blog, read report and code example for AGI/LLM related knowledge.
☆19Updated 3 months ago
Related projects ⓘ
Alternatives and complementary repositories for AGI-Study
- Awesome-LLM-KV-Cache: A curated list of 📙Awesome LLM KV Cache Papers with Codes.☆106Updated last week
- Multi-Candidate Speculative Decoding☆28Updated 6 months ago
- ☆154Updated last month
- The Official Implementation of Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference☆38Updated this week
- 📰 Must-read papers on KV Cache Compression (constantly updating 🤗).☆136Updated this week
- Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**☆138Updated 5 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆144Updated 5 months ago
- The related works and background techniques about Openai o1☆142Updated last week
- Puzzles for learning Triton, play it with minimal environment configuration!☆121Updated last week
- ☆64Updated 3 months ago
- Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.☆90Updated 9 months ago
- A collection of 150+ surveys on LLMs☆209Updated last month
- ☆76Updated 4 months ago
- Awesome list for LLM quantization☆127Updated this week
- The official code for paper "parallel speculative decoding with adaptive draft length."☆24Updated 2 months ago
- ☆89Updated 7 months ago
- DeepSpeed教程 & 示例注释 & 学习笔记 (大模型高效训练)☆116Updated last year
- SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration☆26Updated last month
- official code for GliDe with a CaPE☆10Updated 3 months ago
- [NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.☆391Updated 3 months ago
- [Preprint] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models☆50Updated 3 months ago
- MagicPIG: LSH Sampling for Efficient LLM Generation☆58Updated 3 weeks ago
- Inference code for LLaMA models☆109Updated last year
- [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference☆202Updated 2 weeks ago
- Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)☆188Updated 3 weeks ago
- ATC23 AE☆43Updated last year
- USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference☆357Updated this week
- ☆147Updated 4 months ago
- A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of …☆104Updated 4 months ago
- A trend starts from "Chain of Thought Prompting Elicits Reasoning in Large Language Models".☆39Updated last year