lessw2020 / triton_kernels_for_fun_and_profit

Related projects: ⓘ

• stas00 / ml-ways
  ML/DL Math and Method notes
  ☆56Updated 9 months ago
• drisspg / transformer_nuggets
  A place to store reusable transformer components of my own creation or found on the interwebs
  ☆43Updated 3 weeks ago
• mobiusml / gemlite
  Simple and fast low-bit matmul kernels in CUDA
  ☆48Updated this week
• siboehm / ShallowSpeed
  Small scale distributed training of sequential deep learning models, built on Numpy and MPI.
  ☆88Updated 11 months ago
• alexzhang13 / Triton-Puzzles-Solutions
  Personal solutions to the Triton Puzzles
  ☆11Updated 2 months ago
• AakashKumarNain / mistral_jax
  This is a port of Mistral-7B model in JAX
  ☆29Updated 2 months ago
• srush / triton-autodiff
  Experiment of using Tangent to autodiff triton
  ☆66Updated 7 months ago
• graphcore-research / out-of-the-box-fp8-training
  Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.
  ☆34Updated 2 months ago
• UmerHA / triton_util
  Make triton easier
  ☆39Updated 3 months ago
• cchan / nanoGPT-fp8
  ☆12Updated last year
• joey00072 / ohara
  Collection of autoregressive model implementation
  ☆62Updated 2 weeks ago
• aredden / torch-cublas-hgemm
  PyTorch half precision gemm lib w/ fused optional bias + optional relu/gelu
  ☆25Updated 3 weeks ago
• Jokeren / triton-samples
  ☆22Updated last week
• gau-nernst / learn-cuda
  Learn CUDA with PyTorch
  ☆11Updated last month
• tridao / flash-attention-wheels
  ☆38Updated 9 months ago
• shivance / minbpe.c
  a Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization in pure C.
  ☆21Updated 2 months ago
• rasbt / pytorch-memory-optim
  This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…
  ☆84Updated last year
• gevtushenko / llm.c
  LLM training in simple, raw C/CUDA
  ☆79Updated 4 months ago
• cuda-mode / triton-index
  Cataloging released Triton kernels.
  ☆111Updated 3 weeks ago
• dorjeduck / momograd
  A Learning Journey: Micrograd in Mojo 🔥
  ☆57Updated 3 months ago
• softmax1 / Flash-Attention-Softmax-N
  CUDA and Triton implementations of Flash Attention with SoftmaxN.
  ☆66Updated 3 months ago
• neuralmagic / compressed-tensors
  A safetensors extension to efficiently store sparse quantized tensors on disk
  ☆26Updated this week
• armed-gpt / gpt-blazing
  ☆25Updated this week
• cloneofsimo / min-fsdp
  ☆68Updated 2 months ago
• Deep-Learning-Profiling-Tools / triton-viz
  ☆124Updated last week
• mag- / gpu_benchmark
  Gpu benchmark
  ☆35Updated 2 weeks ago
• unixpickle / learn-ptx
  Learning about CUDA by writing PTX code.
  ☆28Updated 6 months ago
• tanaymeh / mamba-train
  A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM
  ☆46Updated 5 months ago
• lucidrains / grokfast-pytorch
  Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"
  ☆82Updated 3 weeks ago
• herrmann / rustorch
  "PyTorch in Rust"
  ☆16Updated 7 months ago