LUMIA-Group / FourierTransformer

Related projects: ⓘ

• bwconrad / soft-moe
  PyTorch implementation of "From Sparse to Soft Mixtures of Experts"
  ☆38Updated last year
• kyegomez / Griffin
  Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
  ☆48Updated last week
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆86Updated 5 months ago
• test-time-training / ttt-lm-kernels
  Inference Speed Benchmark for Learning to (Learn at Test Time): RNNs with Expressive Hidden States
  ☆33Updated 2 months ago
• badripatro / mamba360
  State Space Models
  ☆55Updated 4 months ago
• PKU-ML / non_neg
  Official Code for ICLR 2024 Paper: Non-negative Contrastive Learning
  ☆36Updated 5 months ago
• vulus98 / Rethinking-attention
  My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing o…
  ☆36Updated 9 months ago
• Shwai-He / SparseAdapter
  Source code of EMNLP 2022 Findings paper "SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters"
  ☆18Updated 5 months ago
• OpenNLPLab / HGRN2
  HGRN2: Gated Linear RNNs with State Expansion
  ☆46Updated last month
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆42Updated last year
• sanagno / adaptively_sparse_attention
  ☆16Updated last year
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆62Updated 11 months ago
• ycjing / Awesome-Model-Merging
  A curated list of Model Merging methods.
  ☆71Updated this week
• Hprairie / Bi-Mamba2
  A Triton Kernel for incorporating Bi-Directionality in Mamba2
  ☆43Updated last week
• goombalab / hydra
  Official implementation of "Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers"
  ☆94Updated last month
• minhoooo1 / CatMAE
  CatMAE
  ☆12Updated 9 months ago
• ruthless-man / Awesome-Learn-from-Model
  Awesome Learn From Model Beyond Fine-Tuning: A Survey
  ☆44Updated 9 months ago
• MambaMixer / M2
  ☆41Updated 5 months ago
• kyegomez / MAGVIT2
  Open source community's implementation of the model from "LANGUAGE MODEL BEATS DIFFUSION — TOKENIZER IS KEY TO VISUAL GENERATION"
  ☆15Updated last week
• OpenNLPLab / HGRN
  [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…
  ☆60Updated 4 months ago
• twinkle0331 / LGTM
  [ACL 2023] Code for paper “Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation”(https://arxiv.org/abs/2305.…
  ☆38Updated last year
• wjxts / RegularizedBN
  ☆20Updated last year
• wutaiqiang / MoSLoRA
  ☆54Updated 2 months ago
• TsinghuaC3I / SoRA
  The source code of the EMNLP 2023 main conference paper: Sparse Low-rank Adaptation of Pre-trained Language Models.
  ☆62Updated 6 months ago
• BaohaoLiao / mefts
  [NeurIPS2023] Make Your Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning
  ☆28Updated last year
• DRSY / EMO
  [ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)
  ☆111Updated 6 months ago
• adymaharana / d2pruning
  ☆25Updated 11 months ago
• krafton-ai / mambaformer-icl
  MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248
  ☆33Updated 3 months ago
• radarFudan / Curse-of-memory
  Curse-of-memory phenomenon of RNNs in sequence modelling
  ☆17Updated this week
• smonsays / hypernetwork-attention
  Official code for the paper "Attention as a Hypernetwork"
  ☆20Updated 2 months ago