Alternatives and similar repositories for ExFlow:

Users that are interested in ExFlow are comparing it to the libraries listed below

• d-matrix-ai / keyformer-llm
  ☆51Updated 9 months ago
• hao-ai-lab / vllm-ltr
  [NeurIPS 2024] Efficient LLM Scheduling by Learning to Rank
  ☆32Updated 2 months ago
• LiuXiaoxuanPKU / OSD
  ☆40Updated last month
• microsoft / ParrotServe
  [OSDI'24] Serving LLM-based Applications Efficiently with Semantic Variable
  ☆134Updated 4 months ago
• LoongServe / LoongServe
  ☆82Updated 2 months ago
• andy-yang-1 / DoubleSparse
  16-fold memory access reduction with nearly no loss
  ☆67Updated 2 months ago
• mit-han-lab / Quest
  [ICML 2024] Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference
  ☆235Updated 2 months ago
• TorchMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆114Updated this week
• zhengzangw / Sequence-Scheduling
  PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
  ☆81Updated last year
• AlibabaResearch / flash-llm
  Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity
  ☆196Updated last year
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆152Updated 6 months ago
• DerrickYLJ / TidalDecode
  TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆26Updated last month
• galeselee / Awesome_LLM_System-PaperList
  Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…
  ☆208Updated last month
• UChi-JCL / CacheGen
  ☆83Updated 3 months ago
• Infini-AI-Lab / MagicDec
  Breaking Throughput-Latency Trade-off for Long Sequences with Speculative Decoding
  ☆106Updated last month
• shadowpa0327 / Palu
  Code for Palu: Compressing KV-Cache with Low-Rank Projection
  ☆63Updated 2 months ago
• hao-ai-lab / MuxServe
  ☆48Updated 7 months ago
• YaoJiayi / CacheBlend
  ☆70Updated 3 weeks ago
• thu-pacman / FasterMoE
  ☆72Updated 2 years ago
• mit-han-lab / Block-Sparse-Attention
  A sparse attention kernel supporting mix sparse patterns
  ☆94Updated 3 months ago
• cat538 / SKVQ
  SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
  ☆13Updated 3 months ago
• pku-liang / ArkVale
  ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)
  ☆24Updated last month
• NJUNLP / MCSD
  Multi-Candidate Speculative Decoding
  ☆34Updated 9 months ago
• graphcore-research / llm-inference-research
  An experimentation platform for LLM inference optimisation
  ☆28Updated 4 months ago
• thu-pacman / SmartMoE-AE
  ATC23 AE
  ☆44Updated last year
• LLMServe / SwiftTransformer
  High performance Transformer implementation in C++.
  ☆98Updated this week
• thu-nics / qllm-eval
  Code Repository of Evaluating Quantized Large Language Models
  ☆112Updated 4 months ago
• microsoft / SeerAttention
  SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
  ☆21Updated last month
• feifeibear / ChituAttention
  Quantized Attention on GPU
  ☆34Updated 2 months ago
• FasterDecoding / SnapKV
  ☆212Updated 8 months ago