SeanNaren / min-LLM
Minimal code to train a Large Language Model (LLM).
☆168Updated 2 years ago
Alternatives and similar repositories for min-LLM:
Users that are interested in min-LLM are comparing it to the libraries listed below
- Exploring finetuning public checkpoints on filter 8K sequences on Pile☆115Updated 2 years ago
- Experiments with generating opensource language model assistants☆97Updated last year
- ☆67Updated 2 years ago
- Code repository for the c-BTM paper☆106Updated last year
- Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*☆82Updated last year
- Multipack distributed sampler for fast padding-free training of LLMs☆188Updated 8 months ago
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- ☆97Updated last year
- Experiments on speculative sampling with Llama models☆125Updated last year
- Pre-training code for Amber 7B LLM☆166Updated 11 months ago
- Techniques used to run BLOOM at inference in parallel☆37Updated 2 years ago
- Train very large language models in Jax.☆204Updated last year
- experiments with inference on llama☆104Updated 10 months ago
- ☆92Updated last year
- Finetune Falcon, LLaMA, MPT, and RedPajama on consumer hardware using PEFT LoRA☆103Updated 8 months ago
- Inference code for LLaMA models in JAX☆118Updated 11 months ago
- A crude RLHF layer on top of nanoGPT with Gumbel-Softmax trick☆289Updated last year
- TART: A plug-and-play Transformer module for task-agnostic reasoning☆196Updated last year
- inference code for mixtral-8x7b-32kseqlen☆99Updated last year
- Used for adaptive human in the loop evaluation of language and embedding models.☆309Updated 2 years ago
- Spherical Merge Pytorch/HF format Language Models with minimal feature loss.☆120Updated last year
- Simple and efficient pytorch-native transformer training and inference (batched)☆73Updated last year
- ☆94Updated last year
- Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023☆135Updated 11 months ago
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆93Updated last year
- QLoRA: Efficient Finetuning of Quantized LLMs☆78Updated last year
- batched loras☆341Updated last year
- Pipeline for pulling and processing online language model pretraining data from the web☆177Updated last year
- Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI☆57Updated last year
- A Multilingual Dataset for Parsing Realistic Task-Oriented Dialogs☆114Updated 2 years ago