Alternatives and similar repositories for FusionQuery:

Users that are interested in FusionQuery are comparing it to the libraries listed below

• JingyangXiang / SUBP
  Pytorch implementation of our paper SUBP: Soft Uniform Block Pruning for 1xN Sparse CNNs Multithreading Acceleration accepted by NeurIPS …
  ☆22Updated last year
• tsinghua-ideal / Twilight
  Adaptive Attention Sparsity with Hierarchical Top-p Pruning
  ☆17Updated 2 months ago
• HugoZHL / PQCache
  [SIGMOD 2025] PQCache: Product Quantization-based KVCache for Long Context LLM Inference
  ☆42Updated 2 months ago
• BaiTheBest / SparseLLM
  Official Repo for SparseLLM: Global Pruning of LLMs (NeurIPS 2024)
  ☆56Updated last month
• wzhuang-xmu / LoSA
  [ICLR 2025] The official pytorch implement of "Dynamic Low-Rank Sparse Adaptation for Large Language Models".
  ☆16Updated last month
• Gaffey / ExCP
  Official implementation of ICML 2024 paper "ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking".
  ☆47Updated 9 months ago
• WukLab / preble
  Stateful LLM Serving
  ☆63Updated last month
• cat538 / SKVQ
  SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
  ☆19Updated 6 months ago
• horrible-dong / DNRT
  [ICLR 2024] Dynamic Neural Response Tuning
  ☆16Updated last month
• FFY0 / AdaKV
  The Official Implementation of Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
  ☆72Updated 3 months ago
• andy-yang-1 / DoubleSparse
  16-fold memory access reduction with nearly no loss
  ☆91Updated last month
• hao-ai-lab / MuxServe
  ☆59Updated 10 months ago
• Aaronhuang-778 / SliM-LLM
  SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
  ☆29Updated 8 months ago
• HarryWu99 / llm_kvcache_sparsity
  Implement some method of LLM KV Cache Sparsity
  ☆32Updated 10 months ago
• zhengzangw / Sequence-Scheduling
  PyTorch implementation of paper "Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline".
  ☆85Updated last year
• SJTU-IPADS / fgnn-artifacts
  FGNN's artifact evaluation (EuroSys 2022)
  ☆17Updated 3 years ago
• sjtu-epcc / Tacker
  Tacker: Tensor-CUDA Core Kernel Fusion for Improving the GPU Utilization while Ensuring QoS
  ☆25Updated 2 months ago
• DerrickYLJ / TidalDecode
  [ICLR 2025] TidalDecode: A Fast and Accurate LLM Decoding with Position Persistent Sparse Attention
  ☆35Updated last week
• Infini-AI-Lab / APE
  ☆22Updated 2 months ago
• machilusZ / FastGen
  This repo contains the source code for: Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
  ☆36Updated 8 months ago
• dywsjtu / apparate
  Artifact for "Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving" [SOSP '24]
  ☆24Updated 5 months ago
• stanford-mast / cedar
  ☆11Updated 6 months ago
• YouAreSpecialToMe / QST
  Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
  ☆43Updated 5 months ago
• VITA-Group / llm-kick
  [ICLR 2024] Jaiswal, A., Gan, Z., Du, X., Zhang, B., Wang, Z., & Yang, Y. Compressing llms: The truth is rarely pure and never simple.
  ☆23Updated last week
• project-etalon / etalon
  LLM Serving Performance Evaluation Harness
  ☆77Updated 2 months ago
• microsoft / chunk-attention
  ☆70Updated last week
• mit-han-lab / Block-Sparse-Attention
  A sparse attention kernel supporting mix sparse patterns
  ☆197Updated 2 months ago
• DD-DuDa / BitDecoding
  A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.
  ☆33Updated last month
• flexflow / flexflow-serve
  FlexFlow Serve: Low-Latency, High-Performance LLM Serving
  ☆34Updated last week
• opengear-project / GEAR
  GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM
  ☆160Updated 9 months ago