Alternatives and similar repositories for moe-pruner:

Users that are interested in moe-pruner are comparing it to the libraries listed below

• 18907305772 / FuseAI
  FuseAI Project
  ☆84Updated 2 months ago
• BaichuanSEED / BaichuanSEED.github.io
  Official Repository for Paper "BaichuanSEED: Sharing the Potential of ExtensivE Data Collection and Deduplication by Introducing a Compet…
  ☆18Updated 7 months ago
• FreedomIntelligence / FastLLM
  Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];
  ☆36Updated last year
• princeton-pli / MeCo
  Code for preprint "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"
  ☆36Updated last week
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆62Updated last year
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆56Updated 11 months ago
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆72Updated last year
• Zyphra / Zyda_processing
  ☆32Updated 9 months ago
• Tomorrowdawn / top_nsigma
  The official code repo and data hub of top_nsigma sampling strategy for LLMs.
  ☆24Updated last month
• kyegomez / LM-Infinite
  Implementation of "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"
  ☆42Updated 4 months ago
• zhaochenyang20 / Prompt2Model-Self-Guide
  SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper
  ☆30Updated 10 months ago
• WENGSYX / LMTuner
  LMTuner: Make the LLM Better for Everyone
  ☆34Updated last year
• SprocketLab / sparse_matrix_fine_tuning
  Official repository for ICML 2024 paper "MoRe Fine-Tuning with 10x Fewer Parameters"
  ☆17Updated 2 weeks ago
• menhguin / minp_paper
  Code Implementation, Evaluations, Documentation, Links and Resources for Min P paper
  ☆28Updated 2 weeks ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆54Updated last week
• Zheng0428 / COIG-Kun
  ☆36Updated 6 months ago
• BBuf / RWKV-World-HF-Tokenizer
  ☆34Updated 8 months ago
• fe1ixxu / CPO_SIMPO
  This repository contains the joint use of CPO and SimPO method for better reference-free preference learning methods.
  ☆52Updated 7 months ago
• thunlp / APB
  ☆25Updated last month
• linxihui / dkernel
  ☆19Updated this week
• hamishivi / automated-instruction-selection
  Exploration of automated dataset selection approaches at large scales.
  ☆34Updated 3 weeks ago
• TIGER-AI-Lab / CritiqueFineTuning
  Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate"
  ☆131Updated last month
• GAIR-NLP / Entropy-ABF
  Official implementation for 'Extending LLMs’ Context Window with 100 Samples'
  ☆75Updated last year
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆115Updated 6 months ago
• princeton-nlp / CEPE
  [ACL 2024] Long-Context Language Modeling with Parallel Encodings
  ☆153Updated 9 months ago
• SalesforceAIResearch / GemFilter
  ☆76Updated 2 months ago
• thepowerfuldeez / OLMo
  My fork os allen AI's OLMo for educational purposes.
  ☆30Updated 3 months ago
• liyucheng09 / llm-compressive
  Longitudinal Evaluation of LLMs via Data Compression
  ☆32Updated 10 months ago
• YuchuanTian / RethinkTinyLM
  [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”
  ☆121Updated 2 months ago
• Bui1dMySea / MemLong
  ☆94Updated 3 months ago