Alternatives and similar repositories for Q-Hitter:

Users that are interested in Q-Hitter are comparing it to the libraries listed below

• pku-liang / ArkVale
  ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction (NIPS'24)
  ☆24Updated last month
• pku-liang / MAGIS
  MAGIS: Memory Optimization via Coordinated Graph Transformation and Scheduling for DNN (ASPLOS'24)
  ☆49Updated 7 months ago
• d-matrix-ai / keyformer-llm
  ☆51Updated 9 months ago
• abhibambhaniya / GenZ-LLM-Analyzer
  LLM Inference analyzer for different hardware platforms
  ☆47Updated last month
• naver-aics / lut-gemm
  ☆47Updated 9 months ago
• clevercool / ANT-Quantization
  ☆85Updated last year
• hgyhungry / alcop-artifact
  ☆21Updated last year
• cornell-zhang / llm-datatypes
  Codebase for ICML'24 paper: Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs
  ☆24Updated 6 months ago
• AIS-SNU / Smart-Infinity
  [HPCA'24] Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System
  ☆38Updated 10 months ago
• snu-comparch / Tender
  Tender: Accelerating Large Language Models via Tensor Decompostion and Runtime Requantization (ISCA'24)
  ☆13Updated 6 months ago
• apuaaChen / EVT_AE
  Artifacts of EVT ASPLOS'24
  ☆23Updated 10 months ago
• uwsampl / sparsetir-artifact
  Repository for artifact evaluation of ASPLOS 2023 paper "SparseTIR: Composable Abstractions for Sparse Compilation in Deep Learning"
  ☆24Updated last year
• cat538 / SKVQ
  SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
  ☆13Updated 3 months ago
• hgyhungry / ShflBW_Sparse_NN
  ☆14Updated 2 years ago
• Shigangli / Magicube
  Magicube is a high-performance library for quantized sparse matrix operations (SpMM and SDDMM) of deep learning on Tensor Cores.
  ☆84Updated 2 years ago
• TiledTensor / TiledLower
  ThrillerFlow is a Dataflow Analysis and Codegen Framework written in Rust.
  ☆14Updated last month
• PrincetonUniversity / LLMCompass
  ☆104Updated 6 months ago
• LeiWang1999 / Stream-k.tvm
  ☆19Updated 3 months ago
• HPMLL / DTC-SpMM_ASPLOS24
  ☆25Updated 6 months ago
• MoE-Inf / awesome-moe-inference
  Curated collection of papers in MoE model inference
  ☆36Updated this week
• mit-han-lab / spatten
  [HPCA'21] SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
  ☆81Updated 4 months ago
• snu-comparch / InfiniGen
  InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)
  ☆100Updated 6 months ago
• uchuhimo / amanda
  ☆17Updated 8 months ago
• hao-ai-lab / MuxServe
  ☆48Updated 7 months ago
• HarryWu99 / llm_kvcache_sparsity
  Implement some method of LLM KV Cache Sparsity
  ☆30Updated 7 months ago
• zyxxmu / cam
  Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference
  ☆29Updated 7 months ago
• ScalingIntelligence / CATS
  ☆23Updated 2 months ago
• microsoft / SparTA
  ☆134Updated 5 months ago
• abdelfattah-lab / BitMoD-HPCA-25
  ☆16Updated last month
• Relaxed-System-Lab / HexGen
  [ICML 2024] Serving LLMs on heterogeneous decentralized clusters.
  ☆16Updated 8 months ago