allenai / OLMo-core
PyTorch building blocks for the OLMo ecosystem
☆197Updated this week
Alternatives and similar repositories for OLMo-core:
Users that are interested in OLMo-core are comparing it to the libraries listed below
- Reproducible, flexible LLM evaluations☆191Updated 3 weeks ago
- An extension of the nanoGPT repository for training small MOE models.☆129Updated last month
- A project to improve skills of large language models☆283Updated this week
- EvaByte: Efficient Byte-level Language Models at Scale☆87Updated last month
- Manage scalable open LLM inference endpoints in Slurm clusters☆254Updated 9 months ago
- code for training & evaluating Contextual Document Embedding models☆180Updated this week
- ☆169Updated 2 months ago
- The HELMET Benchmark☆135Updated last week
- ☆105Updated 4 months ago
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆240Updated this week
- The official evaluation suite and dynamic data release for MixEval.☆235Updated 5 months ago
- LongRoPE is a novel method that can extends the context window of pre-trained LLMs to an impressive 2048k tokens.☆219Updated 8 months ago
- 🌾 OAT: A research-friendly framework for LLM online alignment, including preference learning, reinforcement learning, etc.☆325Updated this week
- Benchmarking LLMs with Challenging Tasks from Real Users☆220Updated 5 months ago
- A simple unified framework for evaluating LLMs☆209Updated last week
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆317Updated 4 months ago
- A framework to study AI models in Reasoning, Alignment, and use of Memory (RAM).☆217Updated 2 weeks ago
- OpenCoconut implements a latent reasoning paradigm where we generate thoughts before decoding.☆171Updated 3 months ago
- Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate"☆137Updated 2 months ago
- minimal GRPO implementation from scratch☆72Updated last month
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆72Updated 8 months ago
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆299Updated last year
- Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"☆385Updated 2 weeks ago
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆230Updated 2 months ago
- Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.☆168Updated last month
- Positional Skip-wise Training for Efficient Context Window Extension of LLMs to Extremely Length (ICLR 2024)☆205Updated 11 months ago
- LongEmbed: Extending Embedding Models for Long Context Retrieval (EMNLP 2024)☆133Updated 5 months ago
- Implementation of paper Data Engineering for Scaling Language Models to 128K Context☆459Updated last year
- Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper☆134Updated 9 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆197Updated 9 months ago