Alternatives and similar repositories for CAMEx

Users that are interested in CAMEx are comparing it to the libraries listed below

• Fsoft-AIC / LibMoE
  LibMoE: A LIBRARY FOR COMPREHENSIVE BENCHMARKING MIXTURE OF EXPERTS IN LARGE LANGUAGE MODELS
  ☆46Updated last month
• Adamdad / rational_kat_cu
  ☆79Updated last year
• jongwooko / distillm-2
  Official PyTorch implementation of DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs (ICML 2025 Oral)
  ☆55Updated 7 months ago
• fangyuan-ksgk / selective-attention-transformer
  Unofficial Implementation of Selective Attention Transformer
  ☆20Updated last year
• nasosger / MuToR
  [NeurIPS '25] Multi-Token Prediction Needs Registers
  ☆26Updated last month
• jaisidhsingh / pytorch-mixtures
  One-stop solutions for Mixture of Experts and Mixture of Depth modules in PyTorch.
  ☆26Updated this week
• TianjinYellow / SPAM-Optimizer
  ☆36Updated 11 months ago
• krafton-ai / mambaformer-icl
  MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248
  ☆58Updated last year
• lucidrains / infini-transformer-pytorch
  Implementation of Infini-Transformer in Pytorch
  ☆112Updated last year
• lucidrains / PEER-pytorch
  Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind
  ☆135Updated 3 months ago
• ml-jku / EVA
  One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation
  ☆46Updated 3 months ago
• nbasyl / DoRA
  Official implementation of "DoRA: Weight-Decomposed Low-Rank Adaptation"
  ☆124Updated last year
• g-luo / vlm_cross_modal_reps
  Official PyTorch Implementation for Vision-Language Models Create Cross-Modal Task Representations, ICML 2025
  ☆32Updated 9 months ago
• mlbio-epfl / joint-inference
  [ICLR 2025] Large (Vision) Language Models are Unsupervised In-Context Learners
  ☆22Updated 8 months ago
• pixeli99 / MixLN
  [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…
  ☆29Updated 6 months ago
• zaydzuhri / softpick-attention
  Landing repository for the paper "Softpick: No Attention Sink, No Massive Activations with Rectified Softmax"
  ☆87Updated 5 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆56Updated 2 years ago
• VijayLingam95 / SVFT
  ☆35Updated last year
• ablghtianyi / ICL_Modular_Arithmetic
  ☆19Updated 10 months ago
• osehmathias / lisa
  LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning
  ☆36Updated last year
• andyjm3 / SLTrain
  SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining (NeurIPS 2024)
  ☆39Updated last year
• uservan / ThinkPO
  ☆17Updated 6 months ago
• ambisinister / lossfreebalance
  toy reproduction of Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
  ☆30Updated last year
• jacobmarks / awesome-neurips-2023
  Conference schedule, top papers, and analysis of the data for NeurIPS 2023!
  ☆120Updated 2 years ago
• chuanyang-Zheng / DAPE
  The this is the official implementation of "DAPE: Data-Adaptive Positional Encoding for Length Extrapolation"
  ☆41Updated last year
• facebookresearch / ModelRatatouille
  Recycling diverse models
  ☆46Updated 3 years ago
• OscarXZQ / weight-selection
  ☆191Updated last year
• deep-spin / adasplash
  AdaSplash: Adaptive Sparse Flash Attention (aka Flash Entmax Attention)
  ☆32Updated 4 months ago
• tianyu-z / VCR
  Official Repo for the paper: VCR: Visual Caption Restoration. Check arxiv.org/pdf/2406.06462 for details.
  ☆32Updated 11 months ago
• AmeenAli / HiddenMambaAttn
  Official PyTorch Implementation of "The Hidden Attention of Mamba Models"
  ☆231Updated 3 months ago