Alternatives and similar repositories for DisTrO:

Users that are interested in DisTrO are comparing it to the libraries listed below

• PrimeIntellect-ai / OpenDiloco
  OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
  ☆478Updated 2 months ago
• PrimeIntellect-ai / prime
  prime is a framework for efficient, globally distributed training of AI models over the internet.
  ☆689Updated this week
• open-thought / reasoning-gym
  procedural reasoning datasets
  ☆541Updated this week
• wbrickner / noise_step
  noise_step: Training in 1.58b With No Gradient Memory
  ☆217Updated 3 months ago
• bloc97 / DeMo
  DeMo: Decoupled Momentum Optimization
  ☆185Updated 4 months ago
• microsoft / GRIN-MoE
  GRadient-INformed MoE
  ☆261Updated 6 months ago
• SakanaAI / evo-memory
  Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.
  ☆301Updated 5 months ago
• huggingface / picotron
  Minimalistic 4D-parallelism distributed training framework for education purpose
  ☆970Updated 3 weeks ago
• NousResearch / Open-Reasoning-Tasks
  A comprehensive repository of reasoning tasks for LLMs (and beyond)
  ☆426Updated 6 months ago
• KellerJordan / Muon
  Muon optimizer: +>30% sample efficiency with <3% wallclock overhead
  ☆539Updated last week
• seal-rg / recurrent-pretraining
  Pretraining code for a large-scale depth-recurrent language model
  ☆709Updated 3 weeks ago
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆774Updated this week
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆622Updated this week
• NousResearch / Hermes-Function-Calling
  ☆846Updated 6 months ago
• microsoft / Samba
  [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling
  ☆857Updated last month
• PaulPauls / llama3_interpretability_sae
  A complete end-to-end pipeline for LLM interpretability with sparse autoencoders (SAEs) using Llama 3.2, written in pure PyTorch and full…
  ☆605Updated last week
• apple / ml-recurrent-drafter
  ☆205Updated 2 months ago
• vllm-project / llm-compressor
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
  ☆1,162Updated this week
• mistralai / mistral-common
  ☆704Updated 2 weeks ago
• SakanaAI / self-adaptive-llms
  A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!
  ☆1,020Updated 2 months ago
• facebookresearch / memory
  Memory layers use a trainable key-value lookup mechanism to add extra parameters to a model without increasing FLOPs. Conceptually, spars…
  ☆311Updated 3 months ago
• huggingface / search-and-learn
  Recipes to scale inference-time compute of open models
  ☆1,048Updated last month
• aidanmclaughlin / AidanBench
  Aidan Bench attempts to measure <big_model_smell> in LLMs.
  ☆288Updated 3 weeks ago
• facebookresearch / blt
  Code for BLT research paper
  ☆1,439Updated this week
• FailSpy / abliterator
  Simple Python library/structure to ablate features in LLMs which are supported by TransformerLens
  ☆441Updated 9 months ago
• xjdr-alt / entropix-local
  smol models are fun too
  ☆91Updated 4 months ago
• vgel / repeng
  A library for making RepE control vectors
  ☆562Updated 2 months ago
• Zyphra / Zamba2
  PyTorch implementation of models from the Zamba2 series.
  ☆178Updated 2 months ago
• ML-GSAI / LLaDA
  Official PyTorch implementation for "Large Language Diffusion Models"
  ☆1,350Updated 3 weeks ago
• open-thought / system-2-research
  System 2 Reasoning Link Collection
  ☆818Updated 2 weeks ago