NCCL communication API layer, and transport layer created from first principles.
☆16Aug 20, 2025Updated last year
Alternatives and similar repositories for NCCL-From-First-Principles
Users that are interested in NCCL-From-First-Principles are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆10Oct 22, 2026Updated last week
- Mini-SGLang is a mini version of SGLang project ().☆13Sep 24, 2025Updated last year
- W4A16 MoE GEMM CUDA kernel. Tuned for H200, B200 & B300.☆13Oct 20, 2026Updated last week
- A lightweight CuTe-based CUDA kernel for DuoAttention, optimized for large language model inference.☆12Dec 25, 2025Updated 10 months ago
- Kernels by MakoraGenerate☆15May 26, 2026Updated 5 months ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- ☆27Feb 27, 2026Updated 8 months ago
- Triton-based Symmetric Memory operators and examples☆113May 15, 2026Updated 5 months ago
- ☆12Nov 21, 2025Updated 11 months ago
- GPU Functional Descriptor for memory access☆35May 24, 2026Updated 5 months ago
- ☆18Jun 19, 2026Updated 4 months ago
- Flash-Flash KDA: H100-optimized Flash Kimi Delta Attention kernels☆24Jul 24, 2026Updated 3 months ago
- DeepSeek-V3.2-Exp DSA Warmup Lightning Indexer training operator based on tilelang☆52Nov 19, 2025Updated 11 months ago
- Learning notes for understanding modern Machine Learning System.☆64Updated this week
- A dynamic binary instrumentation tool for tracing and analyzing GPU kernel instructions.☆84Updated this week
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Some funny cute/cuteDSL code snippets☆35Mar 2, 2026Updated 7 months ago
- Agent-assisted and full-agent reproducibility package for MLSys 2026 FlashInfer AI Kernel Generation Contest submissions: kernels, agent …☆24Jul 21, 2026Updated 3 months ago
- Code and reproducibility artifacts for reconstructing megakernel persistence with CUDA PDL☆26Aug 5, 2026Updated 2 months ago
- ☆19May 9, 2025Updated last year
- Efficient Mask-Aware Distributed Attention for Long-Context Training☆56Oct 16, 2026Updated last week
- Code repository for the SOSP'25 paper DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism.☆23Nov 28, 2025Updated 11 months ago
- Make FP4 on 5090 Great Again☆24Jul 20, 2026Updated 3 months ago
- A Triton-only attention backend for vLLM☆28Jul 14, 2026Updated 3 months ago
- NaiveRT: a self-contained TP8 inference runtime for the Naive N series on eight Blackwell GPUs with UMMA support. One persistent CUDA gra…☆68Sep 27, 2026Updated last month
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Region-level profiling for CUDA kernels with trace, NVBit, CUPTI, NSys, and an interactive Explorer.☆133Apr 17, 2026Updated 6 months ago
- Flash Attention 2 CUDA implementations☆14Apr 29, 2025Updated last year
- Deployment & experimentation of LLM inference at production scale (P/D disaggregation, KV offloading, parallelism, and autoscaling).☆63Sep 26, 2026Updated last month
- ☆48Nov 1, 2025Updated 11 months ago
- ☆68Feb 24, 2026Updated 8 months ago
- ML from scratch in Jax☆12Aug 20, 2025Updated last year
- ☆27Oct 1, 2025Updated last year
- Interactive 3D model of NVIDIA Vera Rubin hardware (superchip, NVL72 compute tray, HGX NVL8) with connection tracing and a guided tour☆16Oct 1, 2026Updated 3 weeks ago
- observability for speculative decoding☆15Aug 23, 2026Updated 2 months ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- HBFSim (HBF Sim): an open-source High-Bandwidth Flash (HBF) simulator. Runs real LLM inference on a real GPU and injects modeled HBF timi…☆64Updated this week
- Anatomy of High-Performance GEMM with Online Fault Tolerance on GPUs☆14Apr 3, 2025Updated last year
- Quantized LLM training in pure CUDA/C++.☆266Aug 31, 2026Updated last month
- ☆32Jul 2, 2025Updated last year
- ☆51Aug 17, 2026Updated 2 months ago
- Skills for writing tilelang and debugging with CUDA toolkits.☆146Updated this week
- a tool like just and entr. A minimal cross-platform, config-driven tool for running commands whenever files change☆15Sep 2, 2026Updated last month