This module defines a type system for distributed training code, based off of JAX's sharding in types, but adapted for the PyTorch ecosystem.
☆46Oct 22, 2026Updated this week
Alternatives and similar repositories for spmd_types
Users that are interested in spmd_types are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An experimental implementation of compiler-driven automatic sharding of models across a given device mesh.☆97Oct 21, 2026Updated last week
- torch_remat fine-grained activation checkpointing API☆23Updated this week
- torchcomms: a modern PyTorch communications API☆402Updated this week
- Chakra et_replay is a tool designed for replaying Chakra Execution Traces (ET) from machine learning models using PyTorch.☆14Updated this week
- MSLK (Meta Superintelligence Labs Kernels) is a collection of PyTorch GPU operator libraries that are designed and optimized for GenAI tr…☆158Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- ☆21Aug 18, 2022Updated 4 years ago
- A storage solution for PyTorch tensors with distributed tensor support.☆90Updated this week
- ☆84Dec 2, 2022Updated 3 years ago
- PyTorch Single Controller☆1,081Updated this week
- Implementation of FixMatch in PyTorch and experimentations☆12Aug 9, 2020Updated 6 years ago
- SpecTrax is a JAX-native library for neural networks and graph learning, built for performance, composability and modularity.☆44Updated this week
- ☆171Dec 27, 2024Updated last year
- Examples of refining software and circuit implementations from their specifications in the style of the Bird-Meertens formalism.☆17Oct 15, 2026Updated last week
- My study note for mlsys☆14Nov 4, 2024Updated last year
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆552Updated this week
- Pytorch routines for (Ker)nel (Mac)hines☆11Oct 10, 2025Updated last year
- SIMD quantization kernels☆93May 29, 2026Updated 4 months ago
- A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.☆975Updated this week
- ☆17Apr 1, 2020Updated 6 years ago
- ☆145May 10, 2026Updated 5 months ago
- Conveniently export torch.compile compiled products into self-contained Python files☆35Jun 5, 2026Updated 4 months ago
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆166Aug 19, 2026Updated 2 months ago
- Autonomous GPU Kernel Generation & Optimization via Deep Agents☆605Sep 8, 2026Updated last month
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- JaxPP is a library for JAX that enables flexible MPMD pipeline parallelism for large-scale LLM training☆83Oct 16, 2026Updated last week
- Standalone commandline CLI tool for compiling Triton kernels☆20Sep 13, 2024Updated 2 years ago
- Triton-based Symmetric Memory operators and examples☆113May 15, 2026Updated 5 months ago
- My submission for the GPUMODE/AMD fp8 mm challenge☆29Jun 4, 2025Updated last year
- A Quirky Assortment of CuTe Kernels☆1,180Oct 15, 2026Updated 2 weeks ago
- My attempt to improve the speed of the newton schulz algorithm, starting from the dion implementation.☆42Apr 30, 2026Updated 5 months ago
- A library to analyze PyTorch traces.☆564Oct 8, 2026Updated 2 weeks ago
- Communication patterns for AI, built on top of NCCL device and host APIs☆71Oct 21, 2026Updated last week
- ☆191Jun 16, 2024Updated 2 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Proving crash safety for systems with layered recovery☆16Apr 1, 2026Updated 6 months ago
- High Performance Grouped GEMM in PyTorch☆30May 10, 2022Updated 4 years ago
- An Optimizer for Nvidia Compilers.☆144Oct 21, 2026Updated last week
- This repository contains the experimental PyTorch native float8 training UX☆226Aug 1, 2024Updated 2 years ago
- Hacks for PyTorch☆19Apr 18, 2023Updated 3 years ago
- Framework for Algorithmic Correctness Testing of Operators☆18Mar 9, 2026Updated 7 months ago
- A Pure JAX AsyncRL stack☆23Sep 7, 2026Updated last month