Alternatives and similar repositories for AutoMoE:

Users that are interested in AutoMoE are comparing it to the libraries listed below

• ThomasScialom / T0_continual_learning
  Adding new tasks to T0 without catastrophic forgetting
  ☆33Updated 2 years ago
• aniquetahir / JORA
  JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)
  ☆33Updated 11 months ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆33Updated 6 months ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆48Updated 2 years ago
• kamanphoebe / Look-into-MoEs
  [NAACL 2025] A Closer Look into Mixture-of-Experts in Large Language Models
  ☆46Updated last month
• uservan / ThinkPO
  ☆16Updated last month
• kyegomez / Reka-Torch
  Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch
  ☆30Updated last week
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆50Updated 6 months ago
• cliang1453 / task-aware-distillation
  Less is More: Task-aware Layer-wise Distillation for Language Model Compression (ICML2023)
  ☆34Updated last year
• mlfoundations / dataset2metadata
  ☆22Updated last year
• twinkle0331 / LGTM
  [ACL 2023] Code for paper “Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation”(https://arxiv.org/abs/2305.…
  ☆38Updated last year
• locuslab / scaling_laws_data_filtering
  ☆64Updated 11 months ago
• joonkeekim / Instructive-Decoding
  Official repository of "Distort, Distract, Decode: Instruction-Tuned Model Can Refine its Response from Noisy Instructions", ICLR 2024 Sp…
  ☆20Updated last year
• srush / LLM-Talk
  ☆48Updated last year
• wangskyGit / passage-sieve
  official repo of AAAI2024 paper Mitigating the Impact of False Negatives in Dense Retrieval with Contrastive Confidence Regularization
  ☆13Updated last year
• shreyansh26 / Attention-Mask-Patterns
  Using FlexAttention to compute attention with different masking patterns
  ☆42Updated 6 months ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆53Updated last month
• IST-DASLab / RoSA
  Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)
  ☆39Updated last year
• formll / resolving-scaling-law-discrepancies
  ☆18Updated 8 months ago
• shoaibahmed / llm_depth_pruning
  Official implementation of the paper: "A deeper look at depth pruning of LLMs"
  ☆14Updated 8 months ago
• GSYfate / knnlm-limits
  Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"
  ☆22Updated 6 months ago
• IST-DASLab / SparseFinetuning
  Repository for Sparse Finetuning of LLMs via modified version of the MosaicML llmfoundry
  ☆40Updated last year
• yidingjiang / ado
  The repository contains code for Adaptive Data Optimization
  ☆20Updated 3 months ago
• hamishivi / automated-instruction-selection
  Exploration of automated dataset selection approaches at large scales.
  ☆33Updated 3 weeks ago
• yale-nlp / refdpo
  ☆15Updated 8 months ago
• HazyResearch / aioli
  Aioli: A unified optimization framework for language model data mixing
  ☆22Updated 2 months ago
• GATECH-EIC / Linearized-LLM
  [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models
  ☆30Updated 9 months ago
• VITA-Group / Junk_DNA_Hypothesis
  [ICML 2024] Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity; Lu Yin*, Ajay Jaiswal*, Shiwei Liu, So…
  ☆16Updated 9 months ago
• JeanKaddour / NoTrainNoGain
  Revisiting Efficient Training Algorithms For Transformer-based Language Models (NeurIPS 2023)
  ☆79Updated last year
• ctlllll / reward_collapse
  ☆27Updated last year