xyjigsaw / LLM-Pretrain-SFTLinks
Scripts of LLM pre-training and fine-tuning (w/wo LoRA, DeepSpeed)
☆80Updated last year
Alternatives and similar repositories for LLM-Pretrain-SFT
Users that are interested in LLM-Pretrain-SFT are comparing it to the libraries listed below
Sorting:
- 🐋 An unofficial implementation of Self-Alignment with Instruction Backtranslation.☆140Updated last month
- Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation☆79Updated 6 months ago
- ☆141Updated last year
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆157Updated 9 months ago
- [NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other mo…☆370Updated 9 months ago
- ☆81Updated last year
- ☆123Updated last year
- InsTag: A Tool for Data Analysis in LLM Supervised Fine-tuning☆261Updated last year
- ☆187Updated last year
- Heuristic filtering framework for RefineCode☆64Updated 2 months ago
- [ICLR 2025] 🧬 RegMix: Data Mixture as Regression for Language Model Pre-training (Spotlight)☆141Updated 3 months ago
- ☆97Updated last year
- Finetuning LLaMA with RLHF (Reinforcement Learning with Human Feedback) based on DeepSpeed Chat☆115Updated 2 years ago
- 怎么训练一个LLM分词器☆149Updated last year
- LLaMA-TRL: Fine-tuning LLaMA with PPO and LoRA☆218Updated 2 years ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆153Updated 11 months ago
- ☆315Updated last year
- [COLING 2025] ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios☆68Updated 3 weeks ago
- Official Repository for SIGIR2024 Demo Paper "An Integrated Data Processing Framework for Pretraining Foundation Models"☆80Updated 9 months ago
- Fantastic Data Engineering for Large Language Models☆89Updated 5 months ago
- 1st Solution For Conversational Multi-Doc QA Workshop & International Challenge @ WSDM'24 - Xiaohongshu.Inc☆160Updated last year
- Dataset and evaluation script for "Evaluating Hallucinations in Chinese Large Language Models"☆128Updated last year
- Benchmarking Complex Instruction-Following with Multiple Constraints Composition (NeurIPS 2024 Datasets and Benchmarks Track)☆84Updated 3 months ago
- Code implementation of synthetic continued pretraining☆111Updated 5 months ago
- ☆108Updated 6 months ago
- 基于DPO算法微调语言大模型，简单好上手。☆39Updated 11 months ago
- ☆283Updated 10 months ago
- [ACL 2024 Demo] Official GitHub repo for UltraEval: An open source framework for evaluating foundation models.☆243Updated 7 months ago
- ☆142Updated 11 months ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆56Updated last year