Alternatives and similar repositories for AutoMoE:

Users that are interested in AutoMoE are comparing it to the libraries listed below

• HazyResearch / prefix-linear-attention
  ☆47Updated 6 months ago
• IST-DASLab / RoSA
  ☆38Updated 11 months ago
• aniquetahir / JORA
  JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)
  ☆32Updated 8 months ago
• kyegomez / Reka-Torch
  Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch
  ☆28Updated this week
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆40Updated 3 months ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆33Updated 3 months ago
• princeton-nlp / InstructEval
  [NAACL 2024 Findings] Evaluation suite for the systematic evaluation of instruction selection methods.
  ☆23Updated last year
• allenai / easy-to-hard-generalization
  Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"
  ☆46Updated last year
• yifeiwang77 / Self-Correction
  ☆19Updated 2 months ago
• ThomasScialom / T0_continual_learning
  Adding new tasks to T0 without catastrophic forgetting
  ☆32Updated 2 years ago
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆48Updated last year
• kyegomez / Blockwise-Parallel-Transformer
  32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.
  ☆44Updated last year
• yale-nlp / refdpo
  ☆15Updated 5 months ago
• locuslab / scaling_laws_data_filtering
  ☆64Updated 9 months ago
• RAIVNLab / MatFormer-OLMo
  Code repository for the public reproduction of the language modelling experiments on "MatFormer: Nested Transformer for Elastic Inference…
  ☆18Updated last year
• kamanphoebe / Look-into-MoEs
  A Closer Look into Mixture-of-Experts in Large Language Models
  ☆41Updated 5 months ago
• ctlllll / reward_collapse
  ☆26Updated last year
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆53Updated this week
• ZhengxiangShi / InstructionModelling
  [NeurIPS 2024 Main Track] Code for the paper titled "Instruction Tuning With Loss Over Instructions"
  ☆33Updated 7 months ago
• kyegomez / EAOT
  The open source implementation of "Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers"
  ☆19Updated 10 months ago
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆22Updated 7 months ago
• twinkle0331 / LGTM
  [ACL 2023] Code for paper “Tailoring Instructions to Student’s Learning Levels Boosts Knowledge Distillation”(https://arxiv.org/abs/2305.…
  ☆38Updated last year
• formll / resolving-scaling-law-discrepancies
  ☆16Updated 6 months ago
• duykhuongnguyen / LASeR-MAB
  Code for paper: "LASeR: Learning to Adaptively Select Reward Models with Multi-Arm Bandits"
  ☆13Updated 3 months ago
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆25Updated 9 months ago
• nverma1 / merging-text-transformers
  Code for "Merging Text Transformers from Different Initializations"
  ☆19Updated 5 months ago
• OpenNLPLab / LASP
  Linear Attention Sequence Parallelism (LASP)
  ☆74Updated 7 months ago
• uscnlp-lime / safer-instruct
  This is the oficial repository for "Safer-Instruct: Aligning Language Models with Automated Preference Data"
  ☆17Updated 10 months ago
• prateeky2806 / ComPEFT
  ☆25Updated last year