xyjigsaw / LLM-Pretrain-SFT
Scripts of LLM pre-training and fine-tuning (w/wo LoRA, DeepSpeed)
☆74Updated last year
Alternatives and similar repositories for LLM-Pretrain-SFT:
Users that are interested in LLM-Pretrain-SFT are comparing it to the libraries listed below
- Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation☆74Updated 3 months ago
- 🐋 An unofficial implementation of Self-Alignment with Instruction Backtranslation.☆137Updated 8 months ago
- InsTag: A Tool for Data Analysis in LLM Supervised Fine-tuning☆241Updated last year
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆142Updated 5 months ago
- Fantastic Data Engineering for Large Language Models☆71Updated last month
- Finetuning LLaMA with RLHF (Reinforcement Learning with Human Feedback) based on DeepSpeed Chat☆112Updated last year
- 怎么训练一个LLM分词器☆140Updated last year
- [NAACL'24] Self-data filtering of LLM instruction-tuning data using a novel perplexity-based difficulty score, without using any other mo…☆337Updated 5 months ago
- ☆130Updated 10 months ago
- ☆139Updated 7 months ago
- [ACL 2024] FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models☆96Updated 2 months ago
- code for Scaling Laws of RoPE-based Extrapolation☆70Updated last year
- The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.☆64Updated last year
- ☆93Updated last year
- 使用单个24G显卡，从0开始训练LLM☆50Updated 3 months ago
- Official github repo for AutoDetect, an automated weakness detection framework for LLMs.☆41Updated 7 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆153Updated 8 months ago
- Benchmarking Complex Instruction-Following with Multiple Constraints Composition (NeurIPS 2024 Datasets and Benchmarks Track)☆66Updated this week
- ☆80Updated last year
- make LLM easier to use☆59Updated last year
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆38Updated 11 months ago
- ☆121Updated last year
- 基于DPO算法微调语言大模型，简单好上手。☆30Updated 7 months ago
- ☆96Updated 11 months ago
- Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 …☆33Updated 2 months ago
- [SIGIR'24] The official implementation code of MOELoRA.☆146Updated 7 months ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆56Updated 10 months ago
- ☆104Updated 3 months ago
- ☆95Updated 4 months ago
- Dataset and evaluation script for "Evaluating Hallucinations in Chinese Large Language Models"☆118Updated 8 months ago