Alternatives and similar repositories for MT2ST:

Users that are interested in MT2ST are comparing it to the libraries listed below

• NoakLiu / Efficient-Foundation-Models-Survey
  Efficient-Large-Foundation-Model-Inference: A-Perspective-From-Model-and-System-Co-Design [Efficient ML System & Model]
  ☆19Updated 2 weeks ago
• NoakLiu / GraphSnapShot
  GraphSnapShot: Caching Local Structure for Fast Graph Learning [Efficient ML System]
  ☆30Updated 2 months ago
• NoakLiu / LLMEasyQuant
  An Easy-to-Use Toolkit for LLM Quantization on can be executed on Macbook [Efficient ML Model]
  ☆15Updated 3 weeks ago
• henryzhongsc / longctx_bench
  Official implementation for Yuan & Liu & Zhong et al., KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark o…
  ☆61Updated last month
• zhengzangw / Sequence-Scheduling
  PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
  ☆81Updated last year
• DerrickYLJ / TidalDecode
  [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆26Updated last month
• zyxxmu / cam
  Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference
  ☆31Updated 7 months ago
• YouAreSpecialToMe / QST
  Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
  ☆41Updated 2 months ago
• NUS-HPC-AI-Lab / Helen
  The official implementation of "Helen: Optimizing CTR Prediction Models with Frequency-wise Hessian Eigenvalue Regularization"
  ☆15Updated 10 months ago
• TreeAI-Lab / Awesome-KV-Cache-Management
  ☆45Updated 3 weeks ago
• SUSTechBruce / LOOK-M
  [EMNLP 2024 Findings🔥] Official implementation of "LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Infe…
  ☆89Updated 2 months ago
• HKUDS / SepLLM
  SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
  ☆43Updated last month
• hemingkx / SWIFT
  [ICLR 2024] SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
  ☆33Updated last month
• sail-sg / SimLayerKV
  The official implementation of paper: SimLayerKV: A Simple Framework for Layer-Level KV Cache Reduction.
  ☆42Updated 3 months ago
• UChi-JCL / CacheGen
  ☆84Updated 3 months ago
• machilusZ / FastGen
  This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
  ☆32Updated 5 months ago
• zcli-charlie / Awesome-KV-Cache
  ☆51Updated 3 months ago
• bytedance / ShadowKV
  ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
  ☆146Updated 3 months ago
• dilab-zju / self-speculative-decoding
  Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**
  ☆153Updated 8 months ago
• NJUNLP / MCSD
  Multi-Candidate Speculative Decoding
  ☆34Updated 9 months ago
• chenzhao / light-dist-gnn
  ☆46Updated 2 years ago
• andy-yang-1 / DoubleSparse
  16-fold memory access reduction with nearly no loss
  ☆72Updated 2 months ago
• thu-pacman / FasterMoE
  ☆72Updated 2 years ago
• Gaffey / ExCP
  Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".
  ☆44Updated 6 months ago
• graphcore-research / llm-inference-research
  An experimentation platform for LLM inference optimisation
  ☆28Updated 4 months ago
• thunlp / MoEfication
  ☆120Updated 6 months ago
• LiuXiaoxuanPKU / OSD
  ☆40Updated last month
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆77Updated 8 months ago
• FFY0 / AdaKV
  The Official Implementation of Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
  ☆56Updated last week