SeanNaren / min-LLMLinks
Minimal code to train a Large Language Model (LLM).
☆169Updated 2 years ago
Alternatives and similar repositories for min-LLM
Users that are interested in min-LLM are comparing it to the libraries listed below
Sorting:
- Multipack distributed sampler for fast padding-free training of LLMs☆191Updated 10 months ago
- Exploring finetuning public checkpoints on filter 8K sequences on Pile☆115Updated 2 years ago
- Simple implementation of Speculative Sampling in NumPy for GPT-2.☆95Updated last year
- Experiments with generating opensource language model assistants☆97Updated 2 years ago
- Code repository for the c-BTM paper☆106Updated last year
- Scaling Data-Constrained Language Models☆335Updated 9 months ago
- ☆67Updated 2 years ago
- Pre-training code for Amber 7B LLM☆166Updated last year
- batched loras☆343Updated last year
- ☆92Updated last year
- Repository for analysis and experiments in the BigCode project.☆119Updated last year
- Experiments on speculative sampling with Llama models☆128Updated 2 years ago
- experiments with inference on llama☆104Updated last year
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- Pipeline for pulling and processing online language model pretraining data from the web☆178Updated last year
- ☆96Updated 2 years ago
- JAX implementation of the Llama 2 model☆219Updated last year
- Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023☆136Updated last year
- ☆180Updated 2 years ago
- TART: A plug-and-play Transformer module for task-agnostic reasoning☆198Updated 2 years ago
- Inference code for LLaMA models in JAX☆118Updated last year
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆257Updated last year
- Train very large language models in Jax.☆205Updated last year
- Python library which enables complex compositions of language models such as scratchpads, chain of thought, tool use, selection-inference…☆208Updated 3 weeks ago
- Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT☆214Updated 10 months ago
- Techniques used to run BLOOM at inference in parallel☆37Updated 2 years ago
- A bagel, with everything.☆321Updated last year
- Tk-Instruct is a Transformer model that is tuned to solve many NLP tasks by following instructions.☆180Updated 2 years ago
- Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch☆230Updated 9 months ago
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆203Updated last year