PrimeIntellect-ai / OpenDiloco
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
☆317Updated last month
Related projects ⓘ
Alternatives and complementary repositories for OpenDiloco
- prime (previously called ZeroBand) is a framework for efficient, globally distributed training of AI models over the internet.☆212Updated this week
- Distributed Training Over-The-Internet☆688Updated 2 months ago
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆154Updated last month
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆685Updated this week
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆523Updated this week
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆173Updated 4 months ago
- scalable and robust tree-based speculative decoding algorithm☆315Updated 3 months ago
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆262Updated last year
- ☆470Updated 2 months ago
- Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"☆177Updated last month
- Beyond Language Models: Byte Models are Digital World Simulators☆309Updated 5 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆701Updated last week
- GRadient-INformed MoE☆259Updated last month
- [ICML 2024] CLLMs: Consistency Large Language Models☆353Updated this week
- Long context evaluation for large language models☆190Updated this week
- Inference of Mamba models in pure C☆178Updated 8 months ago
- PyTorch implementation of models from the Zamba2 series.☆158Updated this week
- OLMoE: Open Mixture-of-Experts Language Models☆460Updated this week
- ☆505Updated 3 weeks ago
- A throughput-oriented high-performance serving framework for LLMs☆636Updated 2 months ago
- Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models☆196Updated 6 months ago
- Code for Husky, an open-source language agent that solves complex, multi-step reasoning tasks. Husky v1 addresses numerical, tabular and …☆328Updated 5 months ago
- Minimalistic large language model 3D-parallelism training☆1,260Updated this week
- Fast parallel LLM inference for MLX☆149Updated 4 months ago
- ☆430Updated 10 months ago
- Micro Llama is a small Llama based model with 300M parameters trained from scratch with $500 budget☆126Updated 7 months ago
- EfficientQAT: Efficient Quantization-Aware Training for Large Language Models☆224Updated last month
- A high-throughput and memory-efficient inference and serving engine for LLMs☆253Updated last month
- ☆118Updated 3 months ago
- A compact LLM pretrained in 9 days by using high quality data☆262Updated last month