Alternatives and similar repositories for llama3-8x8b-MoE

Users that are interested in llama3-8x8b-MoE are comparing it to the libraries listed below

• Zheng0428 / COIG-Kun
  ☆36Updated 8 months ago
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆56Updated last year
• FlagOpen / Infinity-Instruct
  ☆46Updated 11 months ago
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Updated last year
• RhapsodyAILab / MiniCPM-V-Embedding
  ☆29Updated 8 months ago
• fairyshine / Seal-Tools
  The source code and dataset mentioned in the paper Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmar…
  ☆48Updated 6 months ago
• yongchao98 / PROMST
  Automatic prompt optimization framework for multi-step agent tasks.
  ☆30Updated 6 months ago
• Academic-Hammer / HammerLLM
  1.4B sLLM for Chinese and English - HammerLLM🔨
  ☆44Updated last year
• zhaochenyang20 / Prompt2Model-Self-Guide
  SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper
  ☆32Updated 11 months ago
• swt-user / DMPO
  ☆39Updated 7 months ago
• DAMO-NLP-SG / CLEX
  [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models
  ☆77Updated last year
• wuhy68 / Parameter-Efficient-MoE
  Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks
  ☆143Updated 7 months ago
• beichao1314 / Open-Llama
  The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.
  ☆65Updated 2 years ago
• Tomorrowdawn / top_nsigma
  The official code repo and data hub of top_nsigma sampling strategy for LLMs.
  ☆24Updated 3 months ago
• TsinghuaC3I / Intuitive-Fine-Tuning
  Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process
  ☆27Updated 9 months ago
• hills-code / open-instruct
  ☆17Updated last year
• tianyi-lab / Superfiltering
  [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
  ☆151Updated 8 months ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆132Updated 11 months ago
• AlphaPav / mem-kk-logic
  On Memorization of Large Language Models in Logical Reasoning
  ☆64Updated last month
• yyDing1 / ScaleQuest
  We introduce ScaleQuest, a scalable, novel and cost-effective data synthesis method to unleash the reasoning capability of LLMs.
  ☆62Updated 6 months ago
• Joanna0123 / character_profiling
  Code and Data for the paper "Evaluating Character Understanding of Large Language Models via Character Profiling from Fictional Works".
  ☆17Updated 9 months ago
• RUCAIBox / QuantizedEmpirical
  ☆14Updated last year
• Alpha-VLLM / WeMix-LLM
  ☆17Updated last year
• yegcjs / mixinglaws
  ☆98Updated 7 months ago
• GAIR-NLP / ReAlign
  Reformatted Alignment
  ☆114Updated 7 months ago
• BestAnHongjun / SentenceVAE
  Enable Next-sentence Prediction for Large Language Models with Faster Speed, Higher Accuracy and Longer Context
  ☆32Updated 9 months ago
• GAIR-NLP / OPO
  ☆49Updated last year
• RUC-GSAI / Llama-3-SynE
  Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 …
  ☆32Updated 5 months ago
• sail-sg / regmix
  [ICLR 2025] 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)
  ☆135Updated 3 months ago
• yangjianxin1 / LongQLoRA
  LongQLoRA: Extent Context Length of LLMs Efficiently
  ☆164Updated last year