Alternatives and similar repositories for qwen2_moe_mergekit:

Users that are interested in qwen2_moe_mergekit are comparing it to the libraries listed below

• beichao1314 / Open-Llama
  The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.
  ☆65Updated last year
• taishan1994 / Llama3.1-Finetuning
  对llama3进行全参微调、lora微调以及qlora微调。
  ☆187Updated 5 months ago
• yuanzhoulvpi2017 / SentenceEmbedding
  ☆105Updated 8 months ago
• heyblackC / BetterMixture-Top1-Solution
  天池算法比赛《BetterMixture - 大模型数据混合挑战赛》的第一名top1解决方案
  ☆27Updated 8 months ago
• yanqiangmiffy / how-to-train-tokenizer
  怎么训练一个LLM分词器
  ☆142Updated last year
• yongzhuo / qwen2-sft
  Qwen1.5-SFT(阿里, Ali), Qwen_Qwen1.5-2B-Chat/Qwen_Qwen1.5-7B-Chat微调(transformers)/LORA(peft)/推理
  ☆55Updated 10 months ago
• taishan1994 / sentencepiece_chinese_bpe
  使用sentencepiece中BPE训练中文词表，并在transformers中进行使用。
  ☆116Updated last year
• cjymz886 / LLM-RAG-QA
  LLM+RAG for QA
  ☆22Updated last year
• cwxndl / LLM
  大语言模型应用：RAG、NL2SQL、聊天机器人、预训练、MOE混合专家模型、微调训练、强化学习、天池数据竞赛
  ☆58Updated last month
• mobvoi / seq-monkey-data
  ☆135Updated 10 months ago
• mdy666 / mdy_triton
  ☆102Updated this week
• hengjiUSTC / learn-llm
  ☆105Updated 4 months ago
• alibaba-damo-academy / SpokenNLP
  A wide variety of research projects developed by the SpokenNLP team of Speech Lab, Alibaba Group.
  ☆115Updated 2 months ago
• hhnqqq / GemmaLongText
  ☆15Updated 11 months ago
• suu990901 / LLaMA-MiLe-Loss
  Code for a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models
  ☆62Updated last month
• akaihaoshuai / baby-llama2-chinese_cybertron
  使用单个24G显卡，从0开始训练LLM
  ☆50Updated 5 months ago
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆56Updated 11 months ago
• IronBeliever / CaR
  Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation
  ☆76Updated 4 months ago
• linjh1118 / Llama3-Chinese-ORPO
  基于Llama3，通过进一步CPT，SFT，ORPO得到的中文版Llama3
  ☆17Updated 11 months ago
• yongzhuo / Qwen-SFT
  阿里通义千问(Qwen-7B-Chat/Qwen-7B), 微调/LORA/推理
  ☆85Updated 10 months ago
• ssbuild / qwen_finetuning
  qwen models finetuning
  ☆93Updated 2 weeks ago
• sugarandgugu / Simple-Trl-Training
  基于DPO算法微调语言大模型，简单好上手。
  ☆31Updated 8 months ago
• owenliang / qwen-dpo
  通义千问的DPO训练
  ☆40Updated 6 months ago
• Glanvery / LLM-Travel
  欢迎来到 "LLM-travel" 仓库！探索大语言模型（LLM）的奥秘 🚀。致力于深入理解、探讨以及实现与大模型相关的各种技术、原理和应用。
  ☆303Updated 8 months ago
• taishan1994 / pytorch-distributed-NLP
  pytorch分布式训练
  ☆64Updated last year
• zysNLP / quickllm
  A repo for update and debug Mixtral-7x8B、MOE、ChatGLM3、LLaMa2、 BaChuan、Qwen an other LLM models include new models mixtral, mixtral 8x7b, …
  ☆43Updated last week
• TemporaryLoRA / Temp-LoRA
  ☆100Updated 11 months ago
• OvJat / DeepSpeedTutorial
  DeepSpeed Tutorial
  ☆95Updated 7 months ago
• SkyworkAI / Skywork-MoE
  Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models
  ☆129Updated 9 months ago
• sunkx109 / llama
  Inference code for LLaMA models
  ☆118Updated last year