PrimeIntellect-ai / OpenDiloco
OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
☆308Updated last month
Related projects ⓘ
Alternatives and complementary repositories for OpenDiloco
- prime (previously called ZeroBand) is a framework for efficient, globally distributed training of AI models over the internet.☆207Updated this week
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆172Updated 3 months ago
- GRadient-INformed MoE☆258Updated last month
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆664Updated this week
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆498Updated last week
- PyTorch implementation of models from the Zamba2 series.☆158Updated this week
- ☆467Updated 2 months ago
- Distributed Training Over-The-Internet☆683Updated 2 months ago
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆154Updated 3 weeks ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆250Updated last month
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆261Updated last year
- Fast parallel LLM inference for MLX☆145Updated 4 months ago
- Long context evaluation for large language models☆187Updated last week
- [ICML 2024] CLLMs: Consistency Large Language Models☆351Updated this week
- A comprehensive repository of reasoning tasks for LLMs (and beyond)☆278Updated last month
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆292Updated 10 months ago
- A compact LLM pretrained in 9 days by using high quality data☆260Updated last month
- Beyond Language Models: Byte Models are Digital World Simulators☆309Updated 5 months ago
- An Open Source Toolkit For LLM Distillation☆354Updated last month
- Manage scalable open LLM inference endpoints in Slurm clusters☆236Updated 4 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆697Updated last week
- ☆503Updated 2 weeks ago
- A repository for research on medium sized language models.☆479Updated last week
- OLMoE: Open Mixture-of-Experts Language Models☆438Updated last week
- EfficientQAT: Efficient Quantization-Aware Training for Large Language Models☆222Updated last month
- scalable and robust tree-based speculative decoding algorithm☆313Updated 3 months ago
- ☆116Updated 2 months ago
- ☆148Updated 3 months ago
- Code for "LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding", ACL 2024☆206Updated 2 weeks ago
- An easy-to-understand framework for LLM samplers that rewind and revise generated tokens☆105Updated 2 weeks ago