This module defines a type system for distributed training code, based off of JAX's sharding in types, but adapted for the PyTorch ecosystem.
☆46Nov 14, 2026Updated this week
Alternatives and similar repositories for spmd_types
Users that are interested in spmd_types are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An experimental implementation of compiler-driven automatic sharding of models across a given device mesh.☆102Nov 10, 2026Updated last week
- Local and global SPMD type system supporting erasure☆10Apr 24, 2026Updated 6 months ago
- torch_remat fine-grained activation checkpointing API☆26Nov 11, 2026Updated last week
- (deprecated) torchcomms: a modern PyTorch communications API☆405Updated this week
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆554Nov 10, 2026Updated last week
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Pipeline parallelism for the minimalist☆40Aug 6, 2025Updated last year
- A storage solution for PyTorch tensors with distributed tensor support.☆92Nov 9, 2026Updated last week
- Chakra et_replay is a tool designed for replaying Chakra Execution Traces (ET) from machine learning models using PyTorch.☆14Oct 23, 2026Updated 3 weeks ago
- Compact and Agent-Native MoE Training System☆360Updated this week
- MSLK (Meta Superintelligence Labs Kernels) is a collection of PyTorch GPU operator libraries that are designed and optimized for GenAI tr…☆168Updated this week
- ☆84Dec 2, 2022Updated 3 years ago
- PyTorch Single Controller☆1,084Updated this week
- Automatically insert nvtx ranges to PyTorch models☆22Apr 28, 2021Updated 5 years ago
- Implementation of FixMatch in PyTorch and experimentations☆12Aug 9, 2020Updated 6 years ago
- Serverless GPU API endpoints on Runpod - Get Bonus Credits • AdSkip the infrastructure headaches. Auto-scaling, pay-as-you-go, no-ops approach lets you focus on innovating your application.
- ☆21Aug 18, 2022Updated 4 years ago
- ☆172Dec 27, 2024Updated last year
- This is a minimal artifact with state-of-the-art speculative decoding as described in the SwiftSpec paper [ASPLOS' 26].☆14Mar 24, 2026Updated 7 months ago
- SpecTrax is a JAX-native library for neural networks and graph learning, built for performance, composability and modularity.☆44Oct 26, 2026Updated 3 weeks ago
- A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.☆986Updated this week
- My study note for mlsys☆14Nov 4, 2024Updated 2 years ago
- SIMD quantization kernels☆93May 29, 2026Updated 5 months ago
- ☆17Apr 1, 2020Updated 6 years ago
- This repository contains the experimental PyTorch native float8 training UX☆226Aug 1, 2024Updated 2 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Conveniently export torch.compile compiled products into self-contained Python files☆35Jun 5, 2026Updated 5 months ago
- Pytorch routines for (Ker)nel (Mac)hines☆11Oct 10, 2025Updated last year
- Standalone commandline CLI tool for compiling Triton kernels☆20Sep 13, 2024Updated 2 years ago
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆166Aug 19, 2026Updated 2 months ago
- Vine: The BitBlaze Static Analysis Component☆26Sep 27, 2014Updated 12 years ago
- Communication patterns for AI, built on top of NCCL device and host APIs☆82Updated this week
- My submission for the GPUMODE/AMD fp8 mm challenge☆29Jun 4, 2025Updated last year
- JaxPP is a library for JAX that enables flexible MPMD pipeline parallelism for large-scale LLM training☆86Nov 10, 2026Updated last week
- Triton-based Symmetric Memory operators and examples☆114May 15, 2026Updated 6 months ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- My attempt to improve the speed of the newton schulz algorithm, starting from the dion implementation.☆42Apr 30, 2026Updated 6 months ago
- Autonomous GPU Kernel Generation & Optimization via Deep Agents☆627Sep 8, 2026Updated 2 months ago
- A Quirky Assortment of CuTe Kernels☆1,196Oct 30, 2026Updated 2 weeks ago
- A differentiable, self-modifying esolang where gradient descent is an instruction and programs rewrite their own parameters, designed by …☆38Aug 25, 2026Updated 2 months ago
- ☆192Jun 16, 2024Updated 2 years ago
- ☆14Jun 16, 2026Updated 5 months ago
- High Performance Grouped GEMM in PyTorch☆30May 10, 2022Updated 4 years ago
Are these results useful?