SeanNaren / min-LLMLinks
Minimal code to train a Large Language Model (LLM).
☆170Updated 2 years ago
Alternatives and similar repositories for min-LLM
Users that are interested in min-LLM are comparing it to the libraries listed below
Sorting:
- Exploring finetuning public checkpoints on filter 8K sequences on Pile☆116Updated 2 years ago
- Pipeline for pulling and processing online language model pretraining data from the web☆178Updated last year
- Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023☆136Updated last year
- Code repository for the c-BTM paper☆106Updated last year
- Experiments with generating opensource language model assistants☆97Updated 2 years ago
- Techniques used to run BLOOM at inference in parallel☆37Updated 2 years ago
- Multipack distributed sampler for fast padding-free training of LLMs☆195Updated 11 months ago
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆209Updated last year
- ☆96Updated 2 years ago
- Code accompanying the paper Pretraining Language Models with Human Preferences☆182Updated last year
- A (somewhat) minimal library for finetuning language models with PPO on human feedback.☆85Updated 2 years ago
- ☆180Updated 2 years ago
- A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs☆114Updated 2 years ago
- Experiments on speculative sampling with Llama models☆128Updated 2 years ago
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆95Updated last year
- Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*☆85Updated last year
- batched loras☆344Updated last year
- Pre-training code for Amber 7B LLM☆166Updated last year
- Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch☆229Updated 10 months ago
- ☆72Updated 2 years ago
- Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI☆57Updated last year
- Reverse Instructions to generate instruction tuning data with corpus examples☆214Updated last year
- Datasets collection and preprocessings framework for NLP extreme multitask learning☆184Updated last week
- ☆159Updated 2 years ago
- JAX implementation of the Llama 2 model☆219Updated last year
- Scalable PaLM implementation of PyTorch☆190Updated 2 years ago
- Implementation of Reinforcement Learning from Human Feedback (RLHF)☆171Updated 2 years ago
- Scaling Data-Constrained Language Models☆338Updated 3 weeks ago
- some common Huggingface transformers in maximal update parametrization (µP)☆81Updated 3 years ago
- Finetune Falcon, LLaMA, MPT, and RedPajama on consumer hardware using PEFT LoRA☆104Updated 2 months ago