PrimeIntellect-ai / OpenDilocoLinks
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
☆516Updated 6 months ago
Alternatives and similar repositories for OpenDiloco
Users that are interested in OpenDiloco are comparing it to the libraries listed below
Sorting:
- prime is a framework for efficient, globally distributed training of AI models over the internet.☆779Updated last month
- Distributed Training Over-The-Internet☆946Updated 2 months ago
- prime-rl is a codebase for decentralized async RL training at scale☆368Updated this week
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆341Updated 7 months ago
- Pretraining code for a large-scale depth-recurrent language model☆801Updated this week
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆647Updated 2 months ago
- OLMoE: Open Mixture-of-Experts Language Models☆809Updated 4 months ago
- Muon is Scalable for LLM Training☆1,188Updated 3 months ago
- Efficient LLM Inference over Long Sequences☆383Updated 3 weeks ago
- [ICML 2024] CLLMs: Consistency Large Language Models☆396Updated 8 months ago
- Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"☆500Updated last week
- DFloat11: Lossless LLM Compression for Efficient GPU Inference☆446Updated last month
- Long context evaluation for large language models☆220Updated 4 months ago
- scalable and robust tree-based speculative decoding algorithm☆349Updated 5 months ago
- Reference implementation of Megalodon 7B model☆521Updated 2 months ago
- ☆549Updated 10 months ago
- Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse …☆535Updated this week
- Official implementation of Half-Quadratic Quantization (HQQ)☆845Updated last week
- ☆214Updated 5 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆198Updated last year
- Beyond Language Models: Byte Models are Digital World Simulators☆324Updated last year
- Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.☆316Updated 8 months ago
- Minimalistic large language model 3D-parallelism training☆2,034Updated last week
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆244Updated 5 months ago
- ☆483Updated 3 months ago
- Exploring Applications of GRPO☆243Updated last week
- GRadient-INformed MoE☆263Updated 9 months ago
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆318Updated 2 months ago
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆894Updated 2 months ago
- LLM KV cache compression made easy☆538Updated this week