Alternatives and similar repositories for DisTrO

Users that are interested in DisTrO are comparing it to the libraries listed below

• PrimeIntellect-ai / OpenDiloco
  OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
  ☆498Updated 4 months ago
• PrimeIntellect-ai / prime
  prime is a framework for efficient, globally distributed training of AI models over the internet.
  ☆743Updated last week
• NousResearch / atropos
  Atropos is a Language Model Reinforcement Learning Environments framework for collecting and evaluating LLM trajectories through diverse …
  ☆357Updated this week
• huggingface / picotron
  Minimalistic 4D-parallelism distributed training framework for education purpose
  ☆1,464Updated 2 months ago
• SakanaAI / self-adaptive-llms
  A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!
  ☆1,058Updated 3 months ago
• wbrickner / noise_step
  noise_step: Training in 1.58b With No Gradient Memory
  ☆219Updated 4 months ago
• seal-rg / recurrent-pretraining
  Pretraining code for a large-scale depth-recurrent language model
  ☆760Updated last month
• bloc97 / DeMo
  DeMo: Decoupled Momentum Optimization
  ☆186Updated 5 months ago
• microsoft / Samba
  [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
  ☆871Updated 2 weeks ago
• NousResearch / Open-Reasoning-Tasks
  A comprehensive repository of reasoning tasks for LLMs (and beyond)
  ☆439Updated 7 months ago
• open-thought / reasoning-gym
  procedural reasoning datasets
  ☆580Updated this week
• SakanaAI / evo-memory
  Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.
  ☆307Updated 6 months ago
• huggingface / search-and-learn
  Recipes to scale inference-time compute of open models
  ☆1,071Updated last week
• HazyResearch / ThunderKittens
  Tile primitives for speedy kernels
  ☆2,339Updated this week
• tinygrad / open-gpu-kernel-modules
  NVIDIA Linux open GPU with P2P support
  ☆1,142Updated last week
• SakanaAI / continuous-thought-machines
  Continuous Thought Machines, because thought takes time and reasoning is a process.
  ☆492Updated this week
• facebookresearch / memory
  Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…
  ☆327Updated 5 months ago
• karpathy / nano-llama31
  nanoGPT style version of Llama 3.1
  ☆1,367Updated 9 months ago
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆633Updated 3 weeks ago
• xjdr-alt / entropix
  Entropy Based Sampling and Parallel CoT Decoding
  ☆3,368Updated 6 months ago
• google-deepmind / recurrentgemma
  Open weights language model from Google DeepMind, based on Griffin.
  ☆640Updated 2 months ago
• PrimeIntellect-ai / prime-rl
  prime-rl is a codebase for decentralized RL training at scale
  ☆211Updated this week
• facebookresearch / coconut
  Training Large Language Model to Reason in a Continuous Latent Space
  ☆1,109Updated 3 months ago
• open-thought / system-2-research
  System 2 Reasoning Link Collection
  ☆833Updated 2 months ago
• lucidrains / ring-attention-pytorch
  Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
  ☆512Updated 6 months ago
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆810Updated this week
• PsycheFoundation / psyche
  An open infrastructure to democratize and decentralize the development of superintelligence for humanity.
  ☆310Updated this week
• PaulPauls / llama3_interpretability_sae
  A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and full…
  ☆608Updated last month
• huggingface / nanotron
  Minimalistic large language model 3D-parallelism training
  ☆1,870Updated this week
• mistralai / mistral-common
  ☆714Updated last week