argonne-lcf / Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆16Updated this week
Alternatives and similar repositories for Megatron-DeepSpeed:
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below
- This is a repository with examples to run inference endpoints on various ALCF clusters☆17Updated this week
- Cosmic Tagging Network for Neutrino Physics☆13Updated 9 months ago
- ☆36Updated last month
- Material for the SC22 Deep Learning at Scale Tutorial☆40Updated last year
- Benchmark implementation of CosmoFlow in TensorFlow Keras☆21Updated last year
- "wow, that is really fast." - Kyle Gerard Felker☆9Updated 3 years ago
- N-Ways to Multi-GPU Programming☆19Updated last year
- SC24 Deep Learning at Scale Tutorial Material☆30Updated last month
- SC23 Deep Learning at Scale Tutorial Material☆41Updated 6 months ago
- AI Training Series Material☆32Updated 6 months ago
- ALCF Computational Performance Workshop☆37Updated 2 years ago
- ☆17Updated 5 years ago
- The ALCF hosts a regular simulation, data, and learning workshop to help users scale their applications. This repository contains the exa…☆61Updated 4 months ago
- ☆21Updated 4 years ago
- Collection of small examples for running on ALCF resources☆17Updated last month
- Material for the SC21 Deep Learning at Scale Tutorial☆25Updated 2 years ago
- Reference implementations of MLPerf™ HPC training benchmarks☆47Updated last month
- Sparsity support for PyTorch☆35Updated last week
- This is the public repo for the MLPerf DeepCAM climate data segmentation proposal.☆16Updated 3 years ago
- ☆37Updated this week
- Intermediate MPI lesson☆27Updated last year
- A hands-on introduction to tuning GPU kernels using Kernel Tuner https://github.com/KernelTuner/kernel_tuner/☆30Updated 6 months ago
- Train across all your devices, ezpz 🍋☆18Updated this week
- A bunch of kernels that might make stuff slower 😉☆29Updated this week
- Benchmarks☆15Updated this week
- This material contains content on how to profile and optimize simple Pytorch mnist code using NVIDIA Nsight Systems and Pytorch Profiler☆10Updated last year
- extensible collectives library in triton☆84Updated 6 months ago
- JUPITER Benchmark Suite☆17Updated 7 months ago
- ☆14Updated last year
- COCCL: Compression and precision co-aware collective communication library☆22Updated 2 weeks ago