boweiliu / nccl
Optimized primitives for collective multi-GPU communication
☆9Updated 11 months ago
Alternatives and similar repositories for nccl:
Users that are interested in nccl are comparing it to the libraries listed below
- ☆21Updated 2 months ago
- This repository hosts code that supports the testing infrastructure for the PyTorch organization. For example, this repo hosts the logic …☆92Updated this week
- ☆304Updated 8 months ago
- ☆20Updated last year
- ☆30Updated last week
- ☆20Updated 2 weeks ago
- A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind…☆157Updated 5 months ago
- Two implementations of ZeRO-1 optimizer sharding in JAX☆14Updated last year
- A flexible and efficient implementation of Flash Attention 2.0 for JAX, supporting multiple backends (GPU/TPU/CPU) and platforms (Triton/…☆23Updated 2 months ago
- Personal solutions to the Triton Puzzles☆18Updated 9 months ago
- Benchmark suite for LLMs from Fireworks.ai☆70Updated 2 months ago
- Machine Learning eXperiment Utilities☆46Updated 10 months ago
- Pragmatic approach to parsing import profiles for CI's☆11Updated 10 months ago
- A simple library for scaling up JAX programs☆134Updated 6 months ago
- Write a fast kernel and run it on Discord. See how you compare against the best!☆44Updated this week
- ☆16Updated 6 months ago
- NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the …☆151Updated this week
- PyTorch per step fault tolerance (actively under development)☆291Updated this week
- ☆78Updated 10 months ago
- xpk (Accelerated Processing Kit, pronounced x-p-k,) is a software tool to help Cloud developers to orchestrate training jobs on accelerat…☆115Updated this week
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆244Updated this week
- Applied AI experiments and examples for PyTorch☆262Updated last week
- This repository contains the experimental PyTorch native float8 training UX☆224Updated 9 months ago
- NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference☆65Updated 4 months ago
- EquiTriton is a project that seeks to implement high-performance kernels for commonly used building blocks in equivariant neural networks…☆61Updated last month
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆105Updated this week
- Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.☆120Updated this week
- ☆25Updated this week
- ☆40Updated this week
- Experiment of using Tangent to autodiff triton☆78Updated last year