PythonNut / superbpeLinks
Official code release for "SuperBPE: Space Travel for Language Models"
☆50Updated last week
Alternatives and similar repositories for superbpe
Users that are interested in superbpe are comparing it to the libraries listed below
Sorting:
- ☆47Updated last year
- Code for preprint "Metadata Conditioning Accelerates Language Model Pre-training (MeCo)"☆39Updated last month
- Long Context Extension and Generalization in LLMs☆56Updated 8 months ago
- The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models" [AISTATS …☆59Updated 7 months ago
- ☆88Updated this week
- ☆47Updated 9 months ago
- A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.☆68Updated 10 months ago
- This repo is based on https://github.com/jiaweizzhao/GaLore☆28Updated 8 months ago
- The official implementation for Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free☆40Updated 3 weeks ago
- Language models scale reliably with over-training and on downstream tasks☆97Updated last year
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆43Updated last year
- ☆108Updated last year
- ☆52Updated last year
- Organize the Web: Constructing Domains Enhances Pre-Training Data Curation☆52Updated last month
- Codebase for Instruction Following without Instruction Tuning☆34Updated 8 months ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆75Updated last year
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆32Updated 2 months ago
- Randomized Positional Encodings Boost Length Generalization of Transformers☆82Updated last year
- DPO, but faster 🚀☆42Updated 6 months ago
- The simplest implementation of recent Sparse Attention patterns for efficient LLM inference.☆62Updated 4 months ago
- ☆38Updated last year
- ☆126Updated last year
- ☆32Updated last year
- some common Huggingface transformers in maximal update parametrization (µP)☆80Updated 3 years ago
- ☆34Updated 11 months ago
- Python package for serving a local search engine. One command to download and serve a datastore---that's it 😎.☆20Updated this week
- Code and Configs for Asynchronous RLHF: Faster and More Efficient RL for Language Models☆54Updated last month
- ☆54Updated 11 months ago
- A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models☆57Updated 3 months ago
- ☆20Updated last month