This module defines a type system for distributed training code, based off of JAX's sharding in types, but adapted for the PyTorch ecosystem.
☆43Sep 16, 2026Updated this week
Alternatives and similar repositories for spmd_types
Users that are interested in spmd_types are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An experimental implementation of compiler-driven automatic sharding of models across a given device mesh.☆97Updated this week
- torch_remat fine-grained activation checkpointing API☆19Sep 9, 2026Updated last week
- Communication patterns for AI, built on top of NCCL device and host APIs☆58Updated this week
- torchcomms: a modern PyTorch communications API☆396Updated this week
- SpecTrax is a JAX-native library for neural networks and graph learning, built for performance, composability and modularity.☆43Updated this week
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- Conveniently export torch.compile compiled products into self-contained Python files☆34Jun 5, 2026Updated 3 months ago
- My attempt to improve the speed of the newton schulz algorithm, starting from the dion implementation.☆42Apr 30, 2026Updated 4 months ago
- Standalone commandline CLI tool for compiling Triton kernels☆20Sep 13, 2024Updated 2 years ago
- A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.☆945Updated this week
- Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)☆539Aug 28, 2026Updated 3 weeks ago
- Framework for Algorithmic Correctness Testing of Operators☆16Mar 9, 2026Updated 6 months ago
- ☆21Aug 18, 2022Updated 4 years ago
- Autonomous GPU Kernel Generation & Optimization via Deep Agents☆554Sep 8, 2026Updated last week
- Make triton easier☆49Sep 3, 2026Updated 2 weeks ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- PyTorch Single Controller☆1,076Updated this week
- Automatically insert nvtx ranges to PyTorch models☆22Apr 28, 2021Updated 5 years ago
- Tutorials for NVIDIA CUPTI samples☆73Jul 22, 2026Updated last month
- Compact and Agent-Native MoE Training System☆350Updated this week
- A synthetic story narration dataset to study small audio LMs.☆31Jan 21, 2024Updated 2 years ago
- My study note for mlsys☆15Nov 4, 2024Updated last year
- JaxPP is a library for JAX that enables flexible MPMD pipeline parallelism for large-scale LLM training☆83Sep 11, 2026Updated last week
- Fork of Triton repository for OpenXLA uses of the Triton language and compiler☆17Feb 24, 2026Updated 6 months ago
- ☆85Dec 2, 2022Updated 3 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Manage ML configuration with pydantic☆16Mar 18, 2026Updated 5 months ago
- isla coq infrastructure☆22Mar 11, 2025Updated last year
- EmojiNotion – project for CBMI 2025 conference☆14Jun 6, 2025Updated last year
- NVLink microbenchmark with IBM Power8 and NVIDIA P100 GPU - Master Thesis☆11Aug 23, 2017Updated 9 years ago
- Companion website of "MorphQ: Metamorphic Testing of the Qiskit Quantum Computing Platform", accepted at ICSE 23☆10Mar 12, 2025Updated last year
- A codebase for pretraining multi-billion-scale sparse GPTs.☆30Feb 9, 2026Updated 7 months ago
- Nsight Python is a Python kernel profiling interface based on NVIDIA Nsight Tools☆293Updated this week
- ☆16Apr 11, 2024Updated 2 years ago
- Developer blog for PyTorch☆21Sep 2, 2026Updated 2 weeks ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- ☆76Dec 10, 2025Updated 9 months ago
- JAX support for tvm-ffi abi☆28May 14, 2026Updated 4 months ago
- Infiniband verbs performance tests (fork of git://git.openfabrics.org/~grockah/perftest.git)☆21Jan 14, 2016Updated 10 years ago
- AWS Cryptographic Material Providers Library☆17Updated this week
- ☆171Dec 27, 2024Updated last year
- Okapi BM25 with Go☆13Mar 28, 2023Updated 3 years ago
- ☆79May 4, 2021Updated 5 years ago