assafbk / DeciMamba

Related projects: ⓘ

• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆24Updated 5 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆46Updated 3 weeks ago
• HazyResearch / prefix-linear-attention
  ☆41Updated 2 months ago
• sjelassi / transformers_ssm_copy
  ☆22Updated 6 months ago
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆20Updated 2 months ago
• sail-sg / scaling-with-vocab
  📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆52Updated 3 weeks ago
• Zyphra / Zamba2
  PyTorch implementation of models from the Zamba2 series.
  ☆63Updated last month
• goombalab / phi-mamba
  Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mode…
  ☆61Updated this week
• TianduoWang / DPO-ST
  [ACL 2024] Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoning
  ☆24Updated last month
• maximzubkov / fft-scan
  Efficient PScan implementation in PyTorch
  ☆15Updated 8 months ago
• sustcsonglin / mamba-triton
  ☆42Updated 7 months ago
• BlinkDL / LinearAttentionArena
  Here we will test various linear attention designs.
  ☆55Updated 4 months ago
• Cranial-XIX / longhorn
  Official PyTorch Implementation of the Longhorn Deep State Space Model
  ☆35Updated last month
• IdoAmos / not-from-scratch
  ☆22Updated 10 months ago
• OpenNLPLab / HGRN
  [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…
  ☆60Updated 4 months ago
• Shwai-He / LLM-Drop
  The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".
  ☆43Updated last week
• RobertCsordas / moeut
  ☆50Updated last month
• facebookresearch / RAM
  A framework to study AI models in Reasoning, Alignment, and use of Memory (RAM).
  ☆15Updated this week
• berlino / seq_icl
  ☆48Updated 3 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆42Updated last year
• VITA-Group / Data-Efficient-Scaling
  [ICML 2023] "Data Efficient Neural Scaling Law via Model Reusing" by Peihao Wang, Rameswar Panda, Zhangyang Wang
  ☆13Updated 8 months ago
• yegcjs / DiffusionLLM
  Code for paper "Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning"
  ☆59Updated 7 months ago
• formll / resolving-scaling-law-discrepancies
  ☆15Updated 2 months ago
• yale-nlp / refdpo
  ☆13Updated last month
• IST-DASLab / RoSA
  ☆29Updated 7 months ago
• sail-sg / dice
  Official implementation of Bootstrapping Language Models via DPO Implicit Rewards
  ☆33Updated last month
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆28Updated last week
• epfml / schedules-and-scaling
  ☆47Updated 3 months ago
• Itamarzimm / UnifiedImplicitAttnRepr
  ☆30Updated 3 months ago
• srush / mamba-primer
  ☆35Updated 5 months ago