Alternatives and similar repositories for DisTrO:

Users that are interested in DisTrO are comparing it to the libraries listed below

• PrimeIntellect-ai / OpenDiloco
  OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
  ☆455Updated last month
• PrimeIntellect-ai / prime
  prime is a framework for efficient, globally distributed training of AI models over the internet.
  ☆663Updated this week
• microsoft / VPTQ
  VPTQ, A Flexible and Extreme low-bit quantization algorithm
  ☆593Updated this week
• SakanaAI / self-adaptive-llms
  A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!
  ☆977Updated last month
• microsoft / Samba
  Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"
  ☆849Updated last week
• trotsky1997 / MathBlackBox
  ☆1,006Updated 2 months ago
• huggingface / picotron
  Minimalistic 4D-parallelism distributed training framework for education purpose
  ☆872Updated this week
• wbrickner / noise_step
  noise_step: Training in 1.58b With No Gradient Memory
  ☆216Updated 2 months ago
• KellerJordan / Muon
  Muon optimizer: +>30% sample efficiency with <3% wallclock overhead
  ☆434Updated this week
• NousResearch / Open-Reasoning-Tasks
  A comprehensive repository of reasoning tasks for LLMs (and beyond)
  ☆421Updated 5 months ago
• aidanmclaughlin / AidanBench
  Aidan Bench attempts to measure <big_model_smell> in LLMs.
  ☆279Updated last week
• google-deepmind / recurrentgemma
  Open weights language model from Google DeepMind, based on Griffin.
  ☆622Updated last week
• SakanaAI / evo-memory
  Code to train and evaluate Neural Attention Memory Models to obtain universally-applicable memory systems for transformers.
  ☆294Updated 4 months ago
• magicproduct / hash-hop
  Long context evaluation for large language models
  ☆200Updated last week
• NousResearch / Hermes-Function-Calling
  ☆815Updated 5 months ago
• natolambert / rlhf-book
  Textbook on reinforcement learning from human feedback
  ☆466Updated this week
• mobiusml / hqq
  Official implementation of Half-Quadratic Quantization (HQQ)
  ☆760Updated last week
• NVIDIA / RULER
  This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?
  ☆951Updated last month
• mistralai / mistral-common
  ☆679Updated last month
• facebookresearch / MobileLLM
  MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.
  ☆1,255Updated 2 weeks ago
• huggingface / smollm
  Everything about the SmolLM2 and SmolVLM family of models
  ☆1,969Updated last week
• bloc97 / DeMo
  DeMo: Decoupled Momentum Optimization
  ☆181Updated 3 months ago
• codelion / optillm
  Optimizing inference proxy for LLMs
  ☆2,070Updated this week
• huggingface / nanotron
  Minimalistic large language model 3D-parallelism training
  ☆1,630Updated this week
• vllm-project / llm-compressor
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
  ☆1,032Updated this week
• huggingface / search-and-learn
  Recipes to scale inference-time compute of open models
  ☆1,019Updated last week