PrimeIntellect-ai / OpenDilocoLinks
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
☆562Updated last year
Alternatives and similar repositories for OpenDiloco
Users that are interested in OpenDiloco are comparing it to the libraries listed below
Sorting:
- prime is a framework for efficient, globally distributed training of AI models over the internet.☆850Updated 2 months ago
- Distributed Training Over-The-Internet☆975Updated 3 months ago
- Async RL Training at Scale☆1,034Updated this week
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆371Updated last year
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆674Updated 9 months ago
- ☆592Updated last year
- ☆957Updated 3 months ago
- Muon is Scalable for LLM Training☆1,421Updated 6 months ago
- ☆219Updated last year
- Pretraining and inference code for a large-scale depth-recurrent language model☆861Updated last month
- scalable and robust tree-based speculative decoding algorithm☆366Updated last year
- Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse …☆851Updated this week
- Efficient LLM Inference over Long Sequences☆394Updated 7 months ago
- Reference implementation of Megalodon 7B model☆529Updated 8 months ago
- Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch☆549Updated 8 months ago
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆474Updated 3 weeks ago
- DFloat11 [NeurIPS '25]: Lossless Compression of LLMs and DiTs for Efficient GPU Inference☆600Updated 2 months ago
- Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.☆347Updated last year
- [ICML 2024] CLLMs: Consistency Large Language Models☆410Updated last year
- Checkpoint-engine is a simple middleware to update model weights in LLM inference engines☆902Updated this week
- OLMoE: Open Mixture-of-Experts Language Models☆965Updated 4 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆910Updated last month
- Beyond Language Models: Byte Models are Digital World Simulators☆334Updated last year
- 🎯An accuracy-first, highly efficient quantization toolkit for LLMs, designed to minimize quality degradation across Weight-Only Quantiza…☆839Updated this week
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆356Updated last week
- ☆577Updated last year
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆943Updated 2 months ago
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆252Updated last year
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆201Updated last year
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…☆1,180Updated 4 months ago