Alternatives and similar repositories for diloco_simple

Users that are interested in diloco_simple are comparing it to the libraries listed below

• PrimeIntellect-ai / pccl
  PCCL (Prime Collective Communications Library) implements fault tolerant collective communications over IP
  ☆96Updated this week
• PrimeIntellect-ai / INTELLECT-MATH
  A 7B parameter model for mathematical reasoning
  ☆40Updated 5 months ago
• bloc97 / DeMo
  DeMo: Decoupled Momentum Optimization
  ☆189Updated 7 months ago
• HazyResearch / train-tk
  train with kittens!
  ☆61Updated 8 months ago
• PrimeIntellect-ai / prime-vllm
  Modded vLLM to run pipeline parallelism over public networks
  ☆36Updated 2 months ago
• huggingface / kernel-builder
  👷 Build compute kernels
  ☆77Updated this week
• facebookresearch / fastgen
  Simple high-throughput inference library
  ☆120Updated 2 months ago
• AnswerDotAI / cold-compress
  Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…
  ☆138Updated 11 months ago
• google-deepmind / asyncdiloco
  ☆45Updated last year
• JoeLi12345 / nGPT
  an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)
  ☆101Updated 4 months ago
• LeonGuertler / UnstableBaselines
  ☆91Updated this week
• UmerHA / triton_util
  Make triton easier
  ☆47Updated last year
• yandex-research / swarm
  Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"
  ☆141Updated last year
• evanatyourservice / llm-jax
  Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.
  ☆17Updated 4 months ago
• jax-ml / jax-tpu-embedding
  ☆18Updated this week
• stockeh / mlx-grokking
  Grokking on modular arithmetic in less than 150 epochs in MLX
  ☆14Updated 8 months ago
• NolanoOrg / SpectraSuite
  ☆49Updated last year
• xjdr-alt / mla_blog_translation
  ☆13Updated last year
• insuhan / hyper-attn
  ☆81Updated last year
• leloykun / modded-nanogpt
  NanoGPT (124M) quality in 2.67B tokens
  ☆28Updated 3 weeks ago
• character-ai / MuKoe
  ☆54Updated last year
• abacusai / gh200-llm
  Docker image NVIDIA GH200 machines - optimized for vllm serving and hf trainer finetuning
  ☆47Updated 4 months ago
• Zyphra / tree_attention
  Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters
  ☆128Updated 7 months ago
• nebius / kvax
  A FlashAttention implementation for JAX with support for efficient document mask computation and context parallelism.
  ☆128Updated 3 months ago
• gau-nernst / quantized-training
  Explore training for quantized models
  ☆20Updated last week
• athms / mad-lab
  A MAD laboratory to improve AI architecture designs 🧪
  ☆123Updated 7 months ago
• NousResearch / StripedHyenaTrainer
  ☆61Updated last year
• Jaykef / Triton-nanoGPT
  Custom triton kernels for training Karpathy's nanoGPT.
  ☆19Updated 9 months ago
• HazyResearch / lolcats
  Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"
  ☆244Updated 5 months ago
• HazyResearch / based
  Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"
  ☆237Updated last month