PrimeIntellect-ai / OpenDilocoLinks
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
☆546Updated 10 months ago
Alternatives and similar repositories for OpenDiloco
Users that are interested in OpenDiloco are comparing it to the libraries listed below
Sorting:
- Distributed Training Over-The-Internet☆963Updated 3 weeks ago
- Async RL Training at Scale☆749Updated this week
- ☆894Updated last week
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆355Updated 11 months ago
- Checkpoint-engine is a simple middleware to update model weights in LLM inference engines☆820Updated this week
- Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse …☆743Updated this week
- Efficient LLM Inference over Long Sequences☆390Updated 4 months ago
- scalable and robust tree-based speculative decoding algorithm☆361Updated 9 months ago
- LLM KV cache compression made easy☆680Updated this week
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆663Updated 6 months ago
- ☆218Updated 9 months ago
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU.☆701Updated this week
- DFloat11: Lossless LLM Compression for Efficient GPU Inference☆559Updated 2 months ago
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆446Updated this week
- Pretraining and inference code for a large-scale depth-recurrent language model☆843Updated 3 weeks ago
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA (+ more DSLs)☆655Updated this week
- ☆575Updated last year
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…☆1,147Updated last month
- PyTorch-native post-training at scale☆509Updated this week
- Long context evaluation for large language models☆224Updated 8 months ago
- [ICML 2024] CLLMs: Consistency Large Language Models☆405Updated 11 months ago
- noise_step: Training in 1.58b With No Gradient Memory☆222Updated 10 months ago
- Reverse Engineering Gemma 3n: Google's New Edge-Optimized Language Model☆250Updated 5 months ago
- PyTorch building blocks for the OLMo ecosystem☆317Updated this week
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆249Updated 9 months ago
- Scalable toolkit for efficient model reinforcement☆1,009Updated this week
- FlexAttention based, minimal vllm-style inference engine for fast Gemma 2 inference.☆302Updated last week
- ☆453Updated 2 weeks ago
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆154Updated last year