muriloboratto / NCCL
Sample examples of how to call collective operation functions on multi-GPU environments. A simple example of using broadcast, reduce, allGather, reduceScatter and sendRecv operations.
☆28Updated last year
Alternatives and similar repositories for NCCL:
Users that are interested in NCCL are comparing it to the libraries listed below
- ☆78Updated 4 months ago
- ☆66Updated 3 weeks ago
- Matrix Multiply-Accumulate with CUDA and WMMA( Tensor Core)☆122Updated 4 years ago
- Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.☆54Updated 4 months ago
- Benchmark code for the "Online normalizer calculation for softmax" paper☆62Updated 6 years ago
- ☆46Updated 5 years ago
- CUTLASS and CuTe Examples☆35Updated 2 weeks ago
- ☆12Updated 2 weeks ago
- An extension library of WMMA API (Tensor Core API)☆87Updated 6 months ago
- TiledCUDA is a highly efficient kernel template library designed to elevate CUDA C’s level of abstraction for processing tiles.☆174Updated 2 months ago
- CUDA Matrix Multiplication Optimization☆152Updated 5 months ago
- Magicube is a high-performance library for quantized sparse matrix operations (SpMM and SDDMM) of deep learning on Tensor Cores.☆84Updated 2 years ago
- Fast GPU based tensor core reductions☆13Updated 2 years ago
- ☆178Updated 6 months ago
- ☆94Updated last month
- Implementation of TSM2L and TSM2R -- High-Performance Tall-and-Skinny Matrix-Matrix Multiplication Algorithms for CUDA☆32Updated 4 years ago
- ☆81Updated 8 months ago
- NCCL Examples from Official NVIDIA NCCL Developer Guide.☆14Updated 6 years ago
- ☆25Updated 9 months ago
- A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores☆47Updated last year
- High-speed GEMV kernels, at most 2.7x speedup compared to pytorch baseline.☆93Updated 6 months ago
- Optimize GEMM with tensorcore step by step☆19Updated last year
- Instructions, Docker images, and examples for Nsight Compute and Nsight Systems☆130Updated 4 years ago
- Examples of CUDA implementations by Cutlass CuTe☆127Updated last month
- Penn CIS 5650 (GPU Programming and Architecture) Final Project☆26Updated last year
- Standalone Flash Attention v2 kernel without libtorch dependency☆99Updated 4 months ago
- ☆105Updated 10 months ago
- ☆72Updated 2 years ago
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆33Updated 4 months ago
- This project is about convolution operator optimization on GPU, include GEMM based (Implicit GEMM) convolution.☆24Updated 3 weeks ago