microsoft / AutoMoE

Related projects ⓘ

Alternatives and complementary repositories for AutoMoE

• ctlllll / reward_collapse
  ☆26Updated last year
• kamanphoebe / Look-into-MoEs
  A Closer Look into Mixture-of-Experts in Large Language Models
  ☆40Updated 3 months ago
• Leooyii / LCEG
  Long Context Extension and Generalization in LLMs
  ☆39Updated 2 months ago
• yale-nlp / refdpo
  ☆15Updated 3 months ago
• locuslab / scaling_laws_data_filtering
  ☆64Updated 7 months ago
• aniquetahir / JORA
  JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)
  ☆29Updated 6 months ago
• juvi21 / CoPE-cuda
  Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719
  ☆19Updated 5 months ago
• kyegomez / Reka-Torch
  Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch
  ☆29Updated last week
• allenai / data-efficient-finetuning
  Code for paper 'Data-Efficient FineTuning'
  ☆29Updated last year
• HazyResearch / prefix-linear-attention
  ☆45Updated 4 months ago
• OpenNLPLab / LASP
  Linear Attention Sequence Parallelism (LASP)
  ☆64Updated 5 months ago
• allenai / easy-to-hard-generalization
  Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"
  ☆44Updated 10 months ago
• casmlab / NPHardEval
  Repository for NPHardEval, a quantified-dynamic benchmark of LLMs
  ☆48Updated 7 months ago
• felixzli / synthetic_pretraining
  ☆40Updated 2 years ago
• john-hewitt / implicit-ins
  Codebase for Instruction Following without Instruction Tuning
  ☆32Updated last month
• ScalingIntelligence / large_language_monkeys
  ☆55Updated last month
• Princeton-SysML / kNNLM_privacy
  Official implementation of Privacy Implications of Retrieval-Based Language Models (EMNLP 2023). https://arxiv.org/abs/2305.14888
  ☆36Updated 5 months ago
• cambridgeltl / PairS
  Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators (Liu et al.; arXiv preprint arXiv:2403.…
  ☆37Updated 4 months ago
• srush / LLM-Talk
  ☆47Updated 9 months ago
• sail-sg / scaling-with-vocab
  [NeurIPS-2024] 📈 Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies https://arxiv.org/abs/2407.13623
  ☆69Updated last month
• VITA-Group / Random-MoE-as-Dropout
  [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…
  ☆44Updated last year
• xiamengzhou / training_trajectory_analysis
  [ACL 2023]: Training Trajectories of Language Models Across Scales https://arxiv.org/pdf/2212.09803.pdf
  ☆22Updated last year
• princeton-nlp / InstructEval
  [NAACL 2024 Findings] Evaluation suite for the systematic evaluation of instruction selection methods.
  ☆23Updated last year
• martin-wey / CodeUltraFeedback
  CodeUltraFeedback: aligning large language models to coding preferences
  ☆65Updated 4 months ago
• IST-DASLab / RoSA
  ☆35Updated 9 months ago
• maszhongming / ParaKnowTransfer
  Code for "Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective"
  ☆30Updated 6 months ago
• ThomasScialom / T0_continual_learning
  Adding new tasks to T0 without catastrophic forgetting
  ☆30Updated 2 years ago
• xiayuqing0622 / customized-flash-attention
  Fast and memory-efficient exact attention
  ☆27Updated last week
• dangxingyu / rnn-icrag
  Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
  ☆24Updated 7 months ago
• uscnlp-lime / safer-instruct
  This is the oficial repository for "Safer-Instruct: Aligning Language Models with Automated Preference Data"
  ☆17Updated 8 months ago