NousResearch / DisTrOLinks
Distributed Training Over-The-Internet
☆955Updated 3 months ago
Alternatives and similar repositories for DisTrO
Users that are interested in DisTrO are comparing it to the libraries listed below
Sorting:
- prime is a framework for efficient, globally distributed training of AI models over the internet.☆805Updated 3 months ago
- OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training☆528Updated 7 months ago
- Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse …☆591Updated last week
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆907Updated 3 months ago
- Open weights language model from Google DeepMind, based on Griffin.☆649Updated 2 months ago
- Aidan Bench attempts to measure <big_model_smell> in LLMs.☆312Updated 2 months ago
- Decentralized RL Training at Scale☆472Updated this week
- noise_step: Training in 1.58b With No Gradient Memory☆220Updated 8 months ago
- ☆864Updated last year
- An open infrastructure to democratize and decentralize the development of superintelligence for humanity.☆459Updated this week
- Official inference library for pre-processing of Mistral models☆784Updated this week
- VPTQ, A Flexible and Extreme low-bit quantization algorithm☆652Updated 4 months ago
- A comprehensive repository of reasoning tasks for LLMs (and beyond)☆450Updated 11 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆868Updated last week
- nanoGPT style version of Llama 3.1☆1,424Updated last year
- Deep learning for dummies. All the practical details and useful utilities that go into working with real models.☆813Updated last month
- A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!☆1,137Updated 6 months ago
- Official repository for the paper "Grokfast: Accelerated Grokking by Amplifying Slow Gradients"☆560Updated last year
- ☆561Updated last year
- System 2 Reasoning Link Collection☆852Updated 5 months ago
- NVIDIA Linux open GPU with P2P support☆1,234Updated 2 months ago
- Minimalistic 4D-parallelism distributed training framework for education purpose☆1,693Updated last month
- A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and full…☆622Updated 5 months ago
- Simple Python library/structure to ablate features in LLMs which are supported by TransformerLens☆503Updated last year
- Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.☆319Updated 10 months ago
- Pretraining and inference code for a large-scale depth-recurrent language model☆822Updated last month
- Testing baseline LLMs performance across various models☆305Updated 3 weeks ago
- procedural reasoning datasets☆1,069Updated this week
- On-device intelligence.☆371Updated 5 months ago
- Minimalistic large language model 3D-parallelism training☆2,164Updated this week