siboehm / ShallowSpeed
Small scale distributed training of sequential deep learning models, built on Numpy and MPI.
☆129Updated last year
Alternatives and similar repositories for ShallowSpeed:
Users that are interested in ShallowSpeed are comparing it to the libraries listed below
- ☆198Updated this week
- Cataloging released Triton kernels.☆213Updated 3 months ago
- A curated collection of resources, tutorials, and best practices for learning and mastering NVIDIA CUTLASS☆156Updated 3 weeks ago
- extensible collectives library in triton☆84Updated last week
- ring-attention experiments☆129Updated 5 months ago
- An implementation of the transformer architecture onto an Nvidia CUDA kernel☆178Updated last year
- ☆153Updated last year
- Applied AI experiments and examples for PyTorch☆256Updated 3 weeks ago
- Fast low-bit matmul kernels in Triton☆285Updated this week
- Experiment of using Tangent to autodiff triton☆78Updated last year
- A repository to unravel the language of GPUs, making their kernel conversations easy to understand☆170Updated last week
- Fastest kernels written from scratch☆220Updated last week
- PTX-Tutorial Written Purely By AIs (Deep Research of Openai and Claude 3.7)☆64Updated 2 weeks ago
- KernelBench: Can LLMs Write GPU Kernels? - Benchmark with Torch -> CUDA problems☆263Updated this week
- seqax = sequence modeling + JAX☆153Updated this week
- ☆30Updated 3 months ago
- This repository contains the experimental PyTorch native float8 training UX☆222Updated 8 months ago
- High-Performance SGEMM on CUDA devices☆89Updated 2 months ago
- Write a fast kernel and run it on Discord. See how you compare against the best!☆38Updated this week
- Solve puzzles. Learn CUDA.☆63Updated last year
- Collection of kernels written in Triton language☆118Updated last week
- Learning about CUDA by writing PTX code.☆125Updated last year
- ☆76Updated 5 months ago
- Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs☆237Updated last week
- ☆163Updated 9 months ago
- 🚀 Collection of components for development, training, tuning, and inference of foundation models leveraging PyTorch native components.☆192Updated this week
- A really tiny autograd engine☆91Updated last year
- A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.☆529Updated last month
- The simplest but fast implementation of matrix multiplication in CUDA.☆34Updated 8 months ago
- ☆87Updated last year