Alternatives and similar repositories for LLM-Pretrain-SFT

Users that are interested in LLM-Pretrain-SFT are comparing it to the libraries listed below

• IronBeliever / CaR
  Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation
  ☆90Updated last year
• yanqiangmiffy / how-to-train-tokenizer
  怎么训练一个LLM分词器
  ☆154Updated 2 years ago
• Spico197 / Humpback
  🐋 An unofficial implementation of Self-Alignment with Instruction Backtranslation.
  ☆137Updated 8 months ago
• CASIA-LM / MoDS
  ☆147Updated last year
• tianyi-lab / Superfiltering
  [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning
  ☆186Updated 7 months ago
• OFA-Sys / InsTag
  InsTag: A Tool for Data Analysis in LLM Supervised Fine-tuning
  ☆284Updated 2 years ago
• Strivin0311 / long-llms-learning
  A repository sharing the literatures about long-context large language models, including the methodologies and the evaluation benchmarks
  ☆272Updated last year
• beichao1314 / Open-Llama
  The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.
  ☆67Updated 2 years ago
• ssbuild / llm_rlhf
  realize the reinforcement learning training for gpt2 llama bloom and so on llm model
  ☆26Updated 2 years ago
• yuleiqin / fantastic-data-engineering
  Fantastic Data Engineering for Large Language Models
  ☆93Updated last year
• l294265421 / alpaca-rlhf
  Finetuning LLaMA with RLHF (Reinforcement Learning with Human Feedback) based on DeepSpeed Chat
  ☆117Updated 2 years ago
• xbmxb / RAG-query-rewriting
  ☆216Updated last year
• ZigeW / data_management_LLM
  Collection of training data management explorations for large language models
  ☆336Updated last year
• HuangLK / transpeeder
  train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism
  ☆224Updated 2 years ago
• RUC-GSAI / Yulan-GARDEN
  Official Repository for SIGIR2024 Demo Paper "An Integrated Data Processing Framework for Pretraining Foundation Models"
  ☆85Updated last year
• zhangzhao219 / WSDM-Cup-2024
  1st Solution For Conversational Multi-Doc QA Workshop & International Challenge @ WSDM'24 - Xiaohongshu.Inc
  ☆162Updated 6 months ago
• YJiangcm / Lion
  [EMNLP 2023] Lion: Adversarial Distillation of Proprietary Large Language Models
  ☆212Updated last year
• ssbuild / llm_finetuning
  Large language Model fintuning bloom , opt , gpt, gpt2 ,llama,llama-2,cpmant and so on
  ☆99Updated last year
• yangjianxin1 / LongQLoRA
  LongQLoRA: Extent Context Length of LLMs Efficiently
  ☆168Updated 2 years ago
• MikeGu721 / EasyLLM
  make LLM easier to use
  ☆59Updated 2 years ago
• sufengniu / RefGPT
  ☆164Updated 2 years ago
• jasonvanf / llama-trl
  LLaMA-TRL: Fine-tuning LLaMA with PPO and LoRA
  ☆237Updated 5 months ago
• tianyi-lab / Cherry_LLM
  [NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other mo…
  ☆414Updated 7 months ago
• hengjiUSTC / learn-llm
  ☆115Updated last year
• cavalierlulu / rag_survey
  ☆125Updated last year
• thu-coai / CritiqueLLM
  ☆147Updated last year
• OpenBMB / UltraEval
  [ACL 2024 Demo] Official GitHub repo for UltraEval: An open source framework for evaluating foundation models.
  ☆254Updated last year
• RUC-GSAI / YuLan-IR
  YuLan-IR: Information Retrieval Boosted LMs
  ☆220Updated last year
• CASIA-LM / ChineseWebText
  ☆184Updated 2 years ago
• hkust-nlp / deita
  Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]
  ☆584Updated last year