axonn-ai / axonn

Related projects: ⓘ

• DachengLi1 / AMP
  (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.
  ☆33Updated last year
• spcl / sten
  Sparsity support for PyTorch
  ☆31Updated 11 months ago
• hao-ai-lab / MuxServe
  ☆30Updated 3 months ago
• facebookresearch / fairring
  Fairring (FAIR + Herring) is a plug-in for PyTorch that provides a process group for distributed training that outperforms NCCL at large …
  ☆61Updated 2 years ago
• hpcaitech / TensorNVMe
  A Python library transfers PyTorch tensors between CPU and NVMe
  ☆92Updated last year
• siboehm / ShallowSpeed
  Small scale distributed training of sequential deep learning models, built on Numpy and MPI.
  ☆88Updated 11 months ago
• Deep-Learning-Profiling-Tools / triton-viz
  ☆124Updated last week
• pytorch-labs / applied-ai
  Applied AI experiments and examples for PyTorch
  ☆123Updated last month
• mlcommons / logging
  MLPerf™ logging library
  ☆30Updated this week
• Shigangli / Chimera
  Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines.
  ☆41Updated 9 months ago
• zhuohan123 / terapipe
  ☆62Updated 3 years ago
• microsoft / DeepSpeed-Kernels
  ☆50Updated 3 months ago
• Dao-AILab / fast-hadamard-transform
  Fast Hadamard transform in CUDA, with a PyTorch interface
  ☆87Updated 3 months ago
• tanyuqian / redco
  NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
  ☆58Updated this week
• stanford-futuredata / stk
  ☆83Updated 3 weeks ago
• RulinShao / FastCkpt
  Python package for rematerialization-aware gradient checkpointing
  ☆22Updated 10 months ago
• exists-forall / striped_attention
  ☆35Updated 10 months ago
• tgale96 / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆41Updated 3 weeks ago
• sandeepkumar-skb / pytorch_custom_op
  End to End steps for adding custom ops in PyTorch.
  ☆18Updated 4 years ago
• cuda-mode / triton-index
  Cataloging released Triton kernels.
  ☆111Updated 3 weeks ago
• TorchMoE / MoE-Infinity
  PyTorch library for cost-effective, fast and easy serving of MoE models.
  ☆90Updated last month
• MDK8888 / vllmini
  A minimal implementation of vllm.
  ☆29Updated last month
• ColfaxResearch / cutlass-kernels
  ☆138Updated 2 months ago
• alibaba / easydist
  Automated Parallelization System and Infrastructure for Multiple Ecosystems
  ☆70Updated last month
• facebookresearch / MODel_opt
  Memory Optimizations for Deep Learning (ICML 2023)
  ☆58Updated 6 months ago
• Distributed-AI / PipeTransformer
  PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021
  ☆54Updated 3 years ago
• DS3Lab / CocktailSGD
  ☆24Updated last year
• jiazhihao / attention_superoptimizer
  An Attention Superoptimizer
  ☆19Updated 4 months ago
• saareliad / FTPipe
  FTPipe and related pipeline model parallelism research.
  ☆41Updated last year
• apple / ml-recurrent-drafter
  ☆61Updated 2 weeks ago