stas00 / ml-ways

Related projects ⓘ

Alternatives and complementary repositories for ml-ways

• rasbt / pytorch-memory-optim
  This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…
  ☆85Updated last year
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆71Updated 9 months ago
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆66Updated this week
• UmerHA / triton_util
  Make triton easier
  ☆41Updated 4 months ago
• luyug / magix
  Supercharge huggingface transformers with model parallelism.
  ☆74Updated last month
• mgmalek / efficient_cross_entropy
  ☆76Updated 5 months ago
• eth-easl / fmengine
  Utilities for Training Very Large Models
  ☆56Updated last month
• lucidrains / pytorch-custom-utils
  Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…
  ☆117Updated 3 months ago
• YuchenJin / llm.c
  LLM training in simple, raw C/CUDA
  ☆12Updated last month
• pytorch-labs / torchfix
  TorchFix - a linter for PyTorch-using code with autofix support
  ☆98Updated last month
• cloneofsimo / min-fsdp
  ☆72Updated 4 months ago
• lucidrains / grokfast-pytorch
  Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"
  ☆84Updated 2 months ago
• epfml / DenseFormer
  ☆76Updated 6 months ago
• rwightman / genalog
  Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…
  ☆42Updated 9 months ago
• EleutherAI / nanoGPT-mup
  The simplest, fastest repository for training/finetuning medium-sized GPTs.
  ☆83Updated last week
• srush / anynp
  Proof-of-concept of global switching between numpy/jax/pytorch in a library.
  ☆18Updated 4 months ago
• xrsrke / pipegoose
  Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
  ☆80Updated 10 months ago
• ClashLuke / tpucare
  Automatically take good care of your preemptible TPUs
  ☆31Updated last year
• crowsonkb / torch-dist-utils
  Utilities for PyTorch distributed
  ☆23Updated last year
• gpu-mode / profiling-cuda-in-torch
  ☆133Updated 9 months ago
• srush / mamba-primer
  ☆35Updated 7 months ago
• AnswerDotAI / cold-compress
  Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…
  ☆86Updated 3 months ago
• sgugger / torchdynamo-tests
  ☆20Updated last year
• chainyo / tensorshare
  🤝 Trade any tensors over the network
  ☆30Updated last year
• haileyschoelkopf / triton-index
  See https://github.com/cuda-mode/triton-index/ instead!
  ☆11Updated 6 months ago
• cat-state / tinypar
  ☆20Updated last year
• softmax1 / Flash-Attention-Softmax-N
  CUDA and Triton implementations of Flash Attention with SoftmaxN.
  ☆66Updated 5 months ago
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆35Updated 3 months ago
• drisspg / transformer_nuggets
  A place to store reusable transformer components of my own creation or found on the interwebs
  ☆43Updated this week