OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
☆563Jan 13, 2025Updated last year
Alternatives and similar repositories for OpenDiloco
Users that are interested in OpenDiloco are comparing it to the libraries listed below
Sorting:
- prime is a framework for efficient, globally distributed training of AI models over the internet.☆850Nov 16, 2025Updated 3 months ago
- Distributed Training Over-The-Internet☆979Oct 14, 2025Updated 4 months ago
- ☆47Jan 18, 2024Updated 2 years ago
- TOPLOC: is a novel method for verifiable inference that enables users to verify that LLM providers are using the correct model configurat…☆52Apr 14, 2025Updated 10 months ago
- ☆137Mar 20, 2025Updated 11 months ago
- Modded vLLM to run pipeline parallelism over public networks☆40May 20, 2025Updated 9 months ago
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆133Dec 3, 2024Updated last year
- Asynchronous P2P communication backend for decentralized pipeline parallelism☆42Jun 9, 2025Updated 8 months ago
- Scaling is a distributed training library and installable dependency designed to scale up neural networks, with a dedicated module for tr…☆66Nov 18, 2025Updated 3 months ago
- DeMo: Decoupled Momentum Optimization☆198Dec 2, 2024Updated last year
- Manage ML configuration with pydantic☆16Jan 25, 2026Updated last month
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆478Feb 3, 2026Updated 3 weeks ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆203Jul 17, 2024Updated last year
- ☆34Sep 10, 2024Updated last year
- An Open Source Toolkit For LLM Distillation☆875Dec 21, 2025Updated 2 months ago
- Solidity contracts for the decentralized Prime Network protocol☆26Jul 6, 2025Updated 7 months ago
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆371Dec 12, 2024Updated last year
- This library supports evaluating disparities in generated image quality, diversity, and consistency between geographic regions.☆20Jun 3, 2024Updated last year
- Minimalistic large language model 3D-parallelism training☆2,569Feb 19, 2026Updated last week
- GRadient-INformed MoE☆264Sep 25, 2024Updated last year
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆250Jan 31, 2025Updated last year
- A 7B parameter model for mathematical reasoning☆42Feb 17, 2025Updated last year
- Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"☆149Dec 11, 2023Updated 2 years ago
- Tools for merging pretrained large language models.☆6,814Jan 26, 2026Updated last month
- A comprehensive repository of reasoning tasks for LLMs (and beyond)☆457Sep 27, 2024Updated last year
- Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.☆2,390Jan 11, 2026Updated last month
- PCCL (Prime Collective Communications Library) implements fault tolerant collective communications over IP☆143Sep 12, 2025Updated 5 months ago
- Minimalistic 4D-parallelism distributed training framework for education purpose☆2,090Aug 26, 2025Updated 6 months ago
- peer-to-peer compute and intelligence network that enables decentralized AI development at scale☆137Nov 10, 2025Updated 3 months ago
- torch implementation of diloco☆22May 31, 2024Updated last year
- Efficient Triton Kernels for LLM Training☆6,162Updated this week
- Implementation of the Pairformer model used in AlphaFold 3☆14Updated this week
- GoldFinch and other hybrid transformer components☆45Jul 20, 2024Updated last year
- Linear Attention Sequence Parallelism (LASP)☆88Jun 4, 2024Updated last year
- [NeurIPS 2024] Goldfish Loss: Mitigating Memorization in Generative LLMs☆94Nov 17, 2024Updated last year
- DataComp for Language Models☆1,419Sep 9, 2025Updated 5 months ago
- A family of open-sourced Mixture-of-Experts (MoE) Large Language Models☆1,660Mar 8, 2024Updated last year
- PyTorch native quantization and sparsity for training and inference☆2,696Updated this week
- PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily wri…☆1,444Feb 21, 2026Updated last week