NousResearch / DisTrOLinks
Distributed Training Over-The-Internet
☆940Updated last month
Alternatives and similar repositories for DisTrO
Users that are interested in DisTrO are comparing it to the libraries listed below
Sorting:
- OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training☆508Updated 5 months ago
- prime is a framework for efficient, globally distributed training of AI models over the internet.☆770Updated last month
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆646Updated 2 months ago
- Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse …☆514Updated this week
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆883Updated last month
- prime-rl is a codebase for decentralized async RL training at scale☆347Updated this week
- Open weights language model from Google DeepMind, based on Griffin.☆641Updated 3 weeks ago
- procedural reasoning datasets☆872Updated last week
- A comprehensive repository of reasoning tasks for LLMs (and beyond)☆448Updated 9 months ago
- ☆902Updated 9 months ago
- A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!☆1,106Updated 4 months ago
- System 2 Reasoning Link Collection☆838Updated 3 months ago
- Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.☆311Updated 8 months ago
- noise_step: Training in 1.58b With No Gradient Memory☆219Updated 6 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆837Updated last week
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.☆801Updated 2 weeks ago
- Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM☆1,518Updated this week
- Optimizing inference proxy for LLMs☆2,574Updated this week
- Minimalistic large language model 3D-parallelism training☆1,942Updated this week
- Minimalistic 4D-parallelism distributed training framework for education purpose☆1,554Updated 3 weeks ago
- An open infrastructure to democratize and decentralize the development of superintelligence for humanity.☆418Updated this week
- Aidan Bench attempts to measure <big_model_smell> in LLMs.☆305Updated this week
- Recipes to scale inference-time compute of open models☆1,097Updated last month
- Mirage: Automatically Generating Fast GPU Kernels without Programming in Triton/CUDA☆1,322Updated this week
- Big & Small LLMs working together☆994Updated this week
- 1.58 Bit LLM on Apple Silicon using MLX☆214Updated last year
- A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and full…☆616Updated 3 months ago
- Tile primitives for speedy kernels☆2,478Updated this week
- NanoGPT (124M) in 3 minutes☆2,699Updated last week
- Pretraining code for a large-scale depth-recurrent language model☆783Updated 2 weeks ago