crazycth / WizardLearner
Pretrain、decay、SFT a CodeLLM from scratch 🧙♂️
☆36Updated 8 months ago
Alternatives and similar repositories for WizardLearner:
Users that are interested in WizardLearner are comparing it to the libraries listed below
- ☆95Updated 9 months ago
- This is a repo for showcasing using MCTS with LLMs to solve gsm8k problems☆46Updated 2 weeks ago
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆128Updated 7 months ago
- 本项目用于大模型数学解题能力方面的数据集合成,模型训练及评测,相关文章记录。☆67Updated 4 months ago
- A visuailzation tool to make deep understaning and easier debugging for RLHF training.☆129Updated 3 weeks ago
- Feeling confused about super alignment? Here is a reading list☆42Updated last year
- ☆55Updated 2 months ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆55Updated 9 months ago
- ☆45Updated 7 months ago
- Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"☆209Updated 3 months ago
- code for Scaling Laws of RoPE-based Extrapolation☆70Updated last year
- Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 …☆31Updated last month
- ☆81Updated 9 months ago
- ☆66Updated last week
- ☆88Updated last month
- ☆45Updated 3 months ago
- Official completion of “Training on the Benchmark Is Not All You Need”.☆28Updated last month
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆53Updated last year
- Hammer: Robust Function-Calling for On-Device Language Models via Function Masking☆47Updated 2 weeks ago
- ☆36Updated 4 months ago
- Curation of resources for LLM mathematical reasoning, most of which are screened by @tongyx361 to ensure high quality and accompanied wit…☆105Updated 6 months ago
- A highly capable 2.4B lightweight LLM using only 1T pre-training data with all details.☆144Updated last week
- ☆78Updated last year
- [ACL 2024] The official codebase for the paper "Self-Distillation Bridges Distribution Gap in Language Model Fine-tuning".☆107Updated 2 months ago
- ☆48Updated last year
- ☆96Updated 2 months ago
- [EMNLP 2024] LongAlign: A Recipe for Long Context Alignment of LLMs☆239Updated last month
- Reformatted Alignment☆113Updated 4 months ago
- [SIGIR'24] The official implementation code of MOELoRA.☆143Updated 6 months ago
- ☆137Updated 6 months ago