tanaymeh / mamba-train

Related projects ⓘ

Alternatives and complementary repositories for mamba-train

• Zyphra / tree_attention
  Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
  ☆104Updated last month
• thepowerfuldeez / OLMo
  My fork os allen AI's OLMo for educational purposes.
  ☆28Updated this week
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆92Updated last month
• TRI-ML / linear_open_lm
  A repository for research on medium sized language models.
  ☆74Updated 5 months ago
• RobertCsordas / moeut
  ☆62Updated 3 months ago
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆67Updated this week
• jxiw / MambaInLlama
  [NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models
  ☆174Updated this week
• euclaise / supertrainer2000
  ☆49Updated 8 months ago
• wdlctc / mini-s
  ☆35Updated 3 weeks ago
• wuhy68 / Parameter-Efficient-MoE
  Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  ☆129Updated 2 months ago
• schwartz-lab-NLP / TOVA
  Token Omission Via Attention
  ☆120Updated last month
• goombalab / phi-mamba
  Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mode…
  ☆78Updated 2 months ago
• hahnyuan / PB-LLM
  PB-LLM: Partially Binarized Large Language Models
  ☆148Updated last year
• GreenBitAI / green-bit-llm
  A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.
  ☆74Updated last month
• Pleias / Various-Finetuning
  Set of scripts to finetune LLMs
  ☆36Updated 7 months ago
• arcee-ai / DAM
  ☆40Updated 2 weeks ago
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆29Updated 6 months ago
• chu-tianxiang / QuIP-for-all
  QuIP quantization
  ☆46Updated 8 months ago
• HanGuo97 / lq-lora
  ☆122Updated 9 months ago
• proger / hippogriff
  Griffin MQA + Hawk Linear RNN Hybrid
  ☆85Updated 6 months ago
• FasterDecoding / BitDelta
  ☆184Updated last month
• lucidrains / PEER-pytorch
  Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind
  ☆112Updated 2 months ago
• jxmorris12 / cde
  code for training & evaluating Contextual Document Embedding models
  ☆117Updated this week
• itsnamgyu / block-transformer
  Block Transformer: Global-to-Local Language Modeling for Fast Inference (Official Code)
  ☆135Updated last month
• HazyResearch / lolcats
  Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"
  ☆177Updated last month
• kyegomez / MoE-Mamba
  Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…
  ☆84Updated last week
• Cornell-RelaxML / qtip
  ☆67Updated last week
• Zyphra / Zamba2
  PyTorch implementation of models from the Zamba2 series.
  ☆158Updated this week
• VITA-Group / WeLore
  From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…
  ☆43Updated 4 months ago
• jzhang38 / LongMamba
  Some preliminary explorations of Mamba's context scaling.
  ☆191Updated 9 months ago