Recreating PyTorch from scratch (C/C++, CUDA, NCCL and Python, with multi-GPU support and automatic differentiation!)
☆167Nov 25, 2025Updated 9 months ago
Alternatives and similar repositories for PyNorch
Users that are interested in PyNorch are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Alex Krizhevsky's original code from Google Code☆204Mar 10, 2016Updated 10 years ago
- UNet diffusion model in pure CUDA☆663Jun 28, 2024Updated 2 years ago
- Implementation of FlashAttention (FA1-FA4) in PyTorch for educational and algorithmic clarity☆225Apr 12, 2026Updated 4 months ago
- Trained a 114 million Parameter LLM from Scratch.☆19Jul 21, 2024Updated 2 years ago
- ☆93Jul 5, 2024Updated 2 years ago
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- LLM training parallelisms (DP, FSDP, TP, PP) in pure C☆29Jan 27, 2026Updated 7 months ago
- Minimalistic 4D-parallelism distributed training framework for education purpose☆2,296Aug 26, 2025Updated last year
- Optimize tensor program fast with Felix, a gradient descent autotuner.☆33Mar 5, 2026Updated 6 months ago
- Comprehensive CUDA tutorials for Maths & ML with examples☆237Jun 11, 2025Updated last year
- Row-wise block scaling for fp8 quantization matrix multiplication. Solution to GPU mode AMD challenge.☆19Feb 9, 2026Updated 6 months ago
- ☆10Jan 28, 2024Updated 2 years ago
- Toolkit for morse code audio generation and training DNNs for automatic morse code recognition☆12Aug 5, 2022Updated 4 years ago
- High Performance FP8 GEMM Kernels for SM89 and later GPUs.☆21Jan 24, 2025Updated last year
- High-Performance FP32 GEMM on CUDA devices☆126Jan 21, 2025Updated last year
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- Fast GPU based tensor core reductions☆12Jan 13, 2023Updated 3 years ago
- 使用 CUDA C++ 实现的 llama 模型推理框架☆64Nov 8, 2024Updated last year
- ☆14May 15, 2023Updated 3 years ago
- ☆19Jan 16, 2025Updated last year
- ☆15Jan 16, 2025Updated last year
- 📑 Dive into Big Model Training☆117Dec 1, 2022Updated 3 years ago
- C Compiler written in Kotlin☆13Apr 19, 2024Updated 2 years ago
- A Easy-to-understand TensorOp Matmul Tutorial☆452Mar 5, 2026Updated 6 months ago
- 使用 cutlass 实现 flash-attention 精简版，具有教学意义☆59Aug 12, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- flash attention tutorial written in python, triton, cuda, cutlass☆534Jan 20, 2026Updated 7 months ago
- rl from zero pretrain, can it be done? yes.☆296Sep 28, 2025Updated 11 months ago
- Soccer toy example simulator used in Reinforcement Learning☆12Mar 11, 2018Updated 8 years ago
- A PyTorch implementation of the GPT-OSS-20B architecture. All components are coded from scratch: RoPE with YaRN, RMSNorm, SwiGLU with cla…☆238Dec 2, 2025Updated 9 months ago
- A collection of reusable, high-performance, well-documented, thorough-tested layers and models in Jax☆23Jun 8, 2025Updated last year
- A C11 compiler for the discrete logic computer☆21Apr 3, 2024Updated 2 years ago
- A simple Python tool to measure the performance of ONNX models.☆27Sep 15, 2024Updated last year
- ☆17Jan 22, 2023Updated 3 years ago
- ☆121Mar 18, 2026Updated 5 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- Experimenting with small language models☆75Jan 16, 2024Updated 2 years ago
- Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O☆596Sep 13, 2025Updated 11 months ago
- Repository for the COLM 2025 paper SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths☆19Jul 10, 2025Updated last year
- A light llama-like llm inference framework based on the triton kernel.☆192Updated this week
- Code repository for the SOSP'25 paper DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism.☆22Nov 28, 2025Updated 9 months ago
- This repo implements Diffusion Transformers(DiT) in PyTorch and provides training and inference code on CelebHQ dataset☆70Jan 6, 2025Updated last year
- Development area for another repo: Learn_Bluespec_and_RISCV_Design☆13Jun 28, 2026Updated 2 months ago