MayDomine / Burst-Attention

Related projects: ⓘ

• MayDomine / Seq1F1B
  Sequence-level 1F1B schedule for LLMs.
  ☆16Updated 3 months ago
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆78Updated 4 months ago
• feifeibear / Odysseus-Transformer
  Odysseus: Playground of LLM Sequence Parallelism
  ☆50Updated 3 months ago
• thunlp / Ouroboros
  Ouroboros: Speculative Decoding with Large Model Enhanced Drafting
  ☆60Updated 6 months ago
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆47Updated 2 weeks ago
• fanshiqing / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆57Updated 2 months ago
• RulinShao / LightSeq
  Official repository for LightSeq: Sequence Level Parallelism for Distributed Training of Long Context Transformers
  ☆183Updated last month
• Equationliu / Kangaroo
  Implementation of Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting
  ☆39Updated 2 months ago
• FasterDecoding / SnapKV
  ☆164Updated 4 months ago
• alibaba / ChatLearn
  ☆105Updated last week
• linfeng93 / BiTA
  An innovative method expediting LLMs via streamlined semi-autoregressive generation and draft verification.
  ☆21Updated 7 months ago
• madsys-dev / deepseekv2-profile
  ☆60Updated last month
• microsoft / chunk-attention
  ☆29Updated 4 months ago
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆134Updated 2 months ago
• feifeibear / long-context-attention
  USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference
  ☆309Updated this week
• Gaffey / ExCP
  Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".
  ☆37Updated 2 months ago
• dilab-zju / self-speculative-decoding
  Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**
  ☆127Updated 3 months ago
• MoFHeka / LLaMA-Megatron
  A LLaMA1/LLaMA12 Megatron implement.
  ☆26Updated 9 months ago
• stanford-futuredata / stk
  ☆83Updated 3 weeks ago
• FasterDecoding / TEAL
  ☆75Updated this week
• FasterDecoding / REST
  REST: Retrieval-Based Speculative Decoding, NAACL 2024
  ☆158Updated 4 months ago
• AlibabaPAI / FLASHNN
  ☆67Updated last week
• ModelTC / awesome-lm-system
  Summary of system papers/frameworks/codes/tools on training or serving large model
  ☆56Updated 9 months ago
• mit-han-lab / lmquant
  ☆102Updated 3 months ago
• bytedance / AffineQuant
  Official implementation of the ICLR 2024 paper AffineQuant
  ☆16Updated 5 months ago
• thu-ml / low-bit-optimizers
  Low-bit optimizers for PyTorch
  ☆109Updated 11 months ago
• mit-han-lab / Quest
  [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
  ☆161Updated 2 months ago
• zankner / Hydra
  ☆32Updated 7 months ago
• liyucheng09 / llm-compressive
  Longitudinal Evaluation of LLMs via Data Compression
  ☆25Updated 3 months ago
• hemingkx / Spec-Bench
  Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)
  ☆156Updated 3 months ago