PrimeIntellect-ai / OpenDilocoLinks
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
☆534Updated 8 months ago
Alternatives and similar repositories for OpenDiloco
Users that are interested in OpenDiloco are comparing it to the libraries listed below
Sorting:
- prime is a framework for efficient, globally distributed training of AI models over the internet.☆826Updated 4 months ago
- Distributed Training Over-The-Internet☆959Updated 4 months ago
- Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…☆343Updated 9 months ago
- Distributed RL Training at Scale☆626Updated this week
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆658Updated 5 months ago
- ☆217Updated 8 months ago
- scalable and robust tree-based speculative decoding algorithm☆358Updated 7 months ago
- ☆567Updated last year
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU.☆638Updated this week
- Efficient LLM Inference over Long Sequences☆391Updated 3 months ago
- Checkpoint-engine is a simple middleware to update model weights in LLM inference engines☆734Updated this week
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆338Updated 4 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆878Updated 2 weeks ago
- OLMoE: Open Mixture-of-Experts Language Models☆872Updated this week
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆201Updated last year
- Long context evaluation for large language models☆222Updated 6 months ago
- Beyond Language Models: Byte Models are Digital World Simulators☆329Updated last year
- DFloat11: Lossless LLM Compression for Efficient GPU Inference☆541Updated last month
- noise_step: Training in 1.58b With No Gradient Memory☆221Updated 9 months ago
- ☆558Updated 10 months ago
- [ICML 2024] CLLMs: Consistency Large Language Models☆404Updated 10 months ago
- A throughput-oriented high-performance serving framework for LLMs☆891Updated last week
- Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse …☆697Updated this week
- An Open Source Toolkit For LLM Distillation☆726Updated 2 months ago
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆404Updated last week
- Pretraining and inference code for a large-scale depth-recurrent language model☆828Updated 3 weeks ago
- GRadient-INformed MoE☆263Updated last year
- Reverse Engineering Gemma 3n: Google's New Edge-Optimized Language Model☆243Updated 3 months ago
- ☆725Updated 2 weeks ago
- [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention…☆1,128Updated last month