Alternatives and similar repositories for MGQA

Users that are interested in MGQA are comparing it to the libraries listed below

• lucidrains / soft-moe-pytorch
  Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch
  ☆340Updated 9 months ago
• OscarXZQ / weight-selection
  ☆191Updated last year
• kyegomez / Mixture-of-Depths
  Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆111Updated last month
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆112Updated last year
• fkodom / soft-mixture-of-experts
  PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)
  ☆81Updated 2 years ago
• WailordHe / DenseSSM
  A repository for DenseSSMs
  ☆88Updated last year
• kyegomez / PaLM2-VAdapter
  Implementation of "PaLM2-VAdapter:" from the multi-modal model paper: "PaLM2-VAdapter: Progressively Aligned Language Model Makes a Stron…
  ☆17Updated last year
• kyegomez / Paper-Implementation-Template
  A simple reproducible template to implement AI research papers
  ☆23Updated last year
• snu-mllab / LayerMerge
  Official PyTorch implementation of "LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging" (ICML 2024)
  ☆30Updated last year
• kyegomez / MC-ViT
  Implementation of the model: "(MC-ViT)" from the paper: "Memory Consolidation Enables Long-Context Video Understanding"
  ☆27Updated last week
• kyegomez / MoE-Mamba
  Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…
  ☆119Updated 2 months ago
• lucidrains / mixture-of-attention
  Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts
  ☆121Updated last year
• sramshetty / mixture-of-depths
  An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
  ☆36Updated last year
• UCDvision / NOLA
  Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"
  ☆57Updated last year
• kyegomez / TTL
  Pytorch Implementation of the paper: "Learning to (Learn at Test Time): RNNs with Expressive Hidden States"
  ☆25Updated 3 weeks ago
• kyegomez / SparseAttention
  Pytorch Implementation of the sparse attention from the paper: "Generating Long Sequences with Sparse Transformers"
  ☆93Updated this week
• kyegomez / MegaVIT
  The open source implementation of the model from "Scaling Vision Transformers to 22 Billion Parameters"
  ☆31Updated this week
• kaiyuyue / nxtp
  PyTorch Implementation of Object Recognition as Next Token Prediction [CVPR'24 Highlight]
  ☆182Updated 8 months ago
• RobertCsordas / moe_attention
  Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"
  ☆102Updated last year
• kyegomez / Blockwise-Parallel-Transformer
  32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.
  ☆49Updated 2 years ago
• berlino / gated_linear_attention
  ☆106Updated last year
• lucidrains / sinkhorn-router-pytorch
  Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or otherwise
  ☆40Updated last year
• lucidrains / zorro-pytorch
  Implementation of Zorro, Masked Multimodal Transformer, in Pytorch
  ☆98Updated 2 years ago
• zhixuan-lin / forgetting-transformer
  [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning
  ☆134Updated 3 weeks ago
• bwconrad / soft-moe
  PyTorch implementation of "From Sparse to Soft Mixtures of Experts"
  ☆67Updated 2 years ago
• ExplainableML / fomo_in_flux
  Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]
  ☆61Updated last year
• fkodom / yet-another-retnet
  A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…
  ☆106Updated 2 years ago
• swaggy-TN / EfficientVLM
  EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning (ACL 2023)
  ☆33Updated 2 years ago
• lucidrains / MaMMUT-pytorch
  Implementation of MaMMUT, a simple vision-encoder text-decoder architecture for multimodal tasks from Google, in Pytorch
  ☆103Updated 2 years ago
• DAMO-NLP-SG / Inf-CLIP
  [CVPR 2025 Highlight] The official CLIP training codebase of Inf-CL: "Breaking the Memory Barrier: Near Infinite Batch Size Scaling for C…
  ☆275Updated 11 months ago