Alternatives and similar repositories for HMT-pytorch:

Users that are interested in HMT-pytorch are comparing it to the libraries listed below

• Zyphra / tree_attention
  Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
  ☆126Updated 4 months ago
• whyNLP / LCKV
  Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance…
  ☆148Updated 2 weeks ago
• HazyResearch / based
  Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"
  ☆232Updated 2 months ago
• xiayuqing0622 / flex_head_fa
  Fast and memory-efficient exact attention
  ☆67Updated last month
• RobertCsordas / moeut
  ☆78Updated 8 months ago
• hahnyuan / PB-LLM
  PB-LLM: Partially Binarized Large Language Models
  ☆151Updated last year
• schwartz-lab-NLP / TOVA
  Token Omission Via Attention
  ☆126Updated 6 months ago
• Lucky-Lance / Expert_Sparsity
  [ACL 2024] Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models
  ☆87Updated 11 months ago
• wdlctc / mini-s
  ☆50Updated 5 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆60Updated last year
• IST-DASLab / QuEST
  Work in progress.
  ☆56Updated 2 weeks ago
• ScalingIntelligence / large_language_monkeys
  ☆89Updated 7 months ago
• HazyResearch / zoology
  Understand and test language model architectures on synthetic tasks.
  ☆192Updated last month
• epfml / dynamic-sparse-flash-attention
  ☆143Updated last year
• PiotrNawrot / nano-sparse-attention
  The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.
  ☆59Updated 3 months ago
• HanGuo97 / lq-lora
  ☆125Updated last year
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆43Updated 7 months ago
• insuhan / hyper-attn
  ☆81Updated last year
• shawntan / stickbreaking-attention
  Stick-breaking attention
  ☆52Updated last month
• DRSY / EasyKV
  Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
  ☆62Updated last year
• hdong920 / GRIFFIN
  ☆36Updated 7 months ago
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆30Updated 10 months ago
• sustcsonglin / linear-attention-and-beyond-slides
  ☆69Updated 2 months ago
• IST-DASLab / MicroAdam
  This repository contains code for the MicroAdam paper.
  ☆18Updated 4 months ago
• shawntan / scattermoe
  Triton-based implementation of Sparse Mixture of Experts.
  ☆210Updated 4 months ago
• GATECH-EIC / ShiftAddLLM
  ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
  ☆106Updated 6 months ago
• jzhang38 / LongMamba
  Some preliminary explorations of Mamba's context scaling.
  ☆212Updated last year
• lucidrains / pause-transformer
  Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount…
  ☆53Updated last year
• IST-DASLab / Sparse-Marlin
  Boosting 4-bit inference kernels with 2:4 Sparsity
  ☆72Updated 7 months ago
• jeffreysijuntan / lloco
  The official repo for "LLoCo: Learning Long Contexts Offline"
  ☆116Updated 10 months ago