gau-nernst / learn-cuda

Related projects: ⓘ

• lessw2020 / transformer_central
  Various transformers for FSDP research
  ☆31Updated last year
• cloneofsimo / min-fsdp
  ☆68Updated 2 months ago
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆66Updated 7 months ago
• gnovack / distributed-training-and-deepspeed
  ☆13Updated last year
• drisspg / transformer_nuggets
  A place to store reusable transformer components of my own creation or found on the interwebs
  ☆43Updated 3 weeks ago
• mobiusml / gemlite
  Simple and fast low-bit matmul kernels in CUDA
  ☆48Updated this week
• cat-state / tinypar
  ☆20Updated last year
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆62Updated 2 weeks ago
• xrsrke / pipegoose
  Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
  ☆77Updated 9 months ago
• thecharlieblake / lovely-llama
  An implementation of the Llama architecture, to instruct and delight
  ☆21Updated last month
• aredden / torch-cublas-hgemm
  PyTorch half precision gemm lib w/ fused optional bias + optional relu/gelu
  ☆25Updated 3 weeks ago
• tspeterkim / mixed-precision-from-scratch
  Mixed precision training from scratch with Tensors and CUDA
  ☆18Updated 4 months ago
• stas00 / ml-ways
  ML/DL Math and Method notes
  ☆56Updated 9 months ago
• siboehm / ShallowSpeed
  Small scale distributed training of sequential deep learning models, built on Numpy and MPI.
  ☆88Updated 11 months ago
• rasbt / pytorch-memory-optim
  This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…
  ☆84Updated last year
• muellerzr / import-timer
  Pragmatic approach to parsing import profiles for CI's
  ☆11Updated 2 months ago
• warner-benjamin / optimi
  Fast, Modern, Memory Efficient, and Low Precision PyTorch Optimizers
  ☆53Updated 2 months ago
• AnswerDotAI / cold-compress
  Cold Compress is a hackable, lightweight, and open-source toolkit for creating and benchmarking cache compression methods built on top of…
  ☆73Updated last month
• andyehrenberg / flaxlm
  ☆27Updated this week
• srush / Tensor-Puzzles-Penzai
  ☆17Updated 4 months ago
• mgmalek / efficient_cross_entropy
  ☆66Updated 3 months ago
• sgugger / torchdynamo-tests
  ☆20Updated last year
• cloneofsimo / min-max-gpt
  Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training
  ☆110Updated 5 months ago
• EleutherAI / lm-eval2
  ☆13Updated this week
• Chillee / lit-llama
  Simple (fast) transformer inference in PyTorch with torch.compile + lit-llama code
  ☆10Updated last year
• cuda-mode / ring-attention
  ring-attention experiments
  ☆89Updated 5 months ago
• AakashKumarNain / mistral_jax
  This is a port of Mistral-7B model in JAX
  ☆29Updated 2 months ago
• euclaise / supertrainer2000
  ☆48Updated 6 months ago
• dshah3 / GPU-Puzzles
  Solve puzzles. Learn CUDA.
  ☆53Updated 9 months ago
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆34Updated 2 months ago