salesforce / jaxformer
Minimal library to train LLMs on TPU in JAX with pjit().
☆285Updated last year
Alternatives and similar repositories for jaxformer:
Users that are interested in jaxformer are comparing it to the libraries listed below
- This is the official code for the paper CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning (Neur…☆531Updated 3 months ago
- Repository for analysis and experiments in the BigCode project.☆118Updated last year
- Fine-tune SantaCoder for Code/Text Generation.☆191Updated 2 years ago
- Used for adaptive human in the loop evaluation of language and embedding models.☆309Updated 2 years ago
- CodeGen2 models for program synthesis☆275Updated last year
- The data processing pipeline for the Koala chatbot language model☆117Updated 2 years ago
- Ongoing research training transformer models at scale☆387Updated 8 months ago
- Fast Inference Solutions for BLOOM☆561Updated 6 months ago
- ☆458Updated last year
- Scaling Data-Constrained Language Models☆334Updated 7 months ago
- Accepted by Transactions on Machine Learning Research (TMLR)☆126Updated 7 months ago
- 🐙 OctoPack: Instruction Tuning Code Large Language Models☆463Updated 3 months ago
- A simulation framework for RLHF and alternatives. Develop your RLHF method without collecting human data.☆807Updated 10 months ago
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- Multipack distributed sampler for fast padding-free training of LLMs☆188Updated 8 months ago
- batched loras☆341Updated last year
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆301Updated last year
- Train very large language models in Jax.☆204Updated last year
- ☆270Updated 2 years ago
- Code for the paper "Efficient Training of Language Models to Fill in the Middle"☆177Updated 2 years ago
- distributed trainer for LLMs☆573Updated 11 months ago
- Inference code for LLaMA models in JAX☆118Updated 11 months ago
- A framework for the evaluation of autoregressive code generation language models.☆934Updated 6 months ago
- A multi-programming language benchmark for LLMs☆243Updated 3 months ago
- ☆237Updated 2 years ago
- Simple next-token-prediction for RLHF☆225Updated last year
- Data and code for "DocPrompting: Generating Code by Retrieving the Docs" @ICLR 2023☆243Updated last year
- Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch☆641Updated 4 months ago
- A hard gym for programming☆153Updated 9 months ago
- OpenAlpaca: A Fully Open-Source Instruction-Following Model Based On OpenLLaMA☆302Updated last year