mlfoundations / open_lmLinks
A repository for research on medium sized language models.
☆509Updated 2 months ago
Alternatives and similar repositories for open_lm
Users that are interested in open_lm are comparing it to the libraries listed below
Sorting:
- Scaling Data-Constrained Language Models☆339Updated last month
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆308Updated last year
- Website for hosting the Open Foundation Models Cheat Sheet.☆267Updated 3 months ago
- ☆536Updated 9 months ago
- Multipack distributed sampler for fast padding-free training of LLMs☆199Updated last year
- Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.☆739Updated 10 months ago
- Manage scalable open LLM inference endpoints in Slurm clusters☆270Updated last year
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆256Updated last year
- RuLES: a benchmark for evaluating rule-following in language models☆230Updated 6 months ago
- Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"☆557Updated 7 months ago
- batched loras☆345Updated last year
- A library with extensible implementations of DPO, KTO, PPO, ORPO, and other human-aware loss functions (HALOs).☆879Updated last month
- A bagel, with everything.☆324Updated last year
- The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction☆388Updated last year
- Inference code for Persimmon-8B☆415Updated last year
- Large Context Attention☆727Updated 7 months ago
- Extend existing LLMs way beyond the original training length with constant memory usage, without retraining☆708Updated last year
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆536Updated 3 months ago
- distributed trainer for LLMs☆578Updated last year
- Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates☆461Updated last year
- DSIR large-scale data selection framework for language model training☆258Updated last year
- Official repository for ORPO☆462Updated last year
- Official PyTorch implementation of QA-LoRA☆138Updated last year
- Scalable toolkit for efficient model alignment☆837Updated 3 weeks ago
- Pre-training code for Amber 7B LLM☆167Updated last year
- An open collection of methodologies to help with successful training of large language models.☆506Updated last year
- Implementation of paper Data Engineering for Scaling Language Models to 128K Context☆470Updated last year
- ☆560Updated last year
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆344Updated 8 months ago
- Evaluation suite for LLMs☆358Updated last month