Alternatives and similar repositories for nanoMPI

Users that are interested in nanoMPI are comparing it to the libraries listed below

• jax-ml / scaling-book
  Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs
  ☆430Updated last week
• rkinas / triton-resources
  A curated list of resources for learning and exploring Triton, OpenAI's programming language for writing efficient GPU code.
  ☆378Updated 4 months ago
• MekkCyber / CutlassAcademy
  A curated collection of resources, tutorials, and best practices for learning and mastering NVIDIA CUTLASS
  ☆196Updated 2 months ago
• MekkCyber / TritonAcademy
  A repository to unravel the language of GPUs, making their kernel conversations easy to understand
  ☆188Updated last month
• unixpickle / learn-ptx
  Learning about CUDA by writing PTX code.
  ☆133Updated last year
• gpu-mode / profiling-cuda-in-torch
  ☆161Updated last year
• EurekaLabsAI / tensor
  The Tensor (or Array)
  ☆437Updated 11 months ago
• pytorch-labs / monarch
  PyTorch Single Controller
  ☆318Updated this week
• ulrichstern / cuda-convnet
  Alex Krizhevsky's original code from Google Code
  ☆194Updated 9 years ago
• BobMcDear / attorch
  A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.
  ☆560Updated this week
• hkproj / triton-flash-attention
  ☆180Updated 6 months ago
• siboehm / ShallowSpeed
  Small scale distributed training of sequential deep learning models, built on Numpy and MPI.
  ☆135Updated last year
• gpu-mode / triton-index
  Cataloging released Triton kernels.
  ☆245Updated 6 months ago
• clu0 / unet.cu
  UNet diffusion model in pure CUDA
  ☆610Updated last year
• ScalingIntelligence / KernelBench
  KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems
  ☆475Updated this week
• Maharshi-Pandya / cudacodes
  Learnings and programs related to CUDA
  ☆411Updated 2 weeks ago
• huggingface / picotron_tutorial
  ☆200Updated 5 months ago
• Deep-Learning-Profiling-Tools / triton-viz
  ☆225Updated this week
• bertmaher / simplegemm
  ☆110Updated 4 months ago
• pytorch / torchft
  Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)
  ☆361Updated this week
• ash-01xor / bpe.c
  Simple Byte pair Encoding mechanism used for tokenization process . written purely in C
  ☆134Updated 8 months ago
• gpu-mode / ring-attention
  ring-attention experiments
  ☆143Updated 9 months ago
• linjames0 / Transformer-CUDA
  An implementation of the transformer architecture onto an Nvidia CUDA kernel
  ☆188Updated last year
• salykova / sgemm.cu
  High-Performance SGEMM on CUDA devices
  ☆97Updated 5 months ago
• salykova / sgemm.c
  Multi-Threaded FP32 Matrix Multiplication on x86 CPUs
  ☆350Updated 2 months ago
• gpu-mode / reference-kernels
  Official Problem Sets / Reference Kernels for the GPU MODE Leaderboard!
  ☆67Updated this week
• huggingface / gpu-fryer
  Where GPUs get cooked 👩‍🍳🔥
  ☆236Updated 4 months ago
• SzymonOzog / GPU_Programming
  ☆63Updated this week
• lucasdelimanogueira / PyNorch
  Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)
  ☆150Updated last year
• SzymonOzog / FastSoftmax
  ☆47Updated 6 months ago