argonne-lcf / Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆16Updated this week
Alternatives and similar repositories for Megatron-DeepSpeed:
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below
- ☆36Updated this week
- Material for the SC22 Deep Learning at Scale Tutorial☆41Updated last year
- Cosmic Tagging Network for Neutrino Physics☆13Updated 10 months ago
- SC24 Deep Learning at Scale Tutorial Material☆32Updated 2 months ago
- This is a repository with examples to run inference endpoints on various ALCF clusters☆19Updated last week
- Collection of small examples for running on ALCF resources☆17Updated 3 weeks ago
- ☆21Updated 4 years ago
- AI Training Series Material☆33Updated 7 months ago
- Benchmark implementation of CosmoFlow in TensorFlow Keras☆21Updated last year
- A lattice QCD library.☆11Updated 2 weeks ago
- Guidelines on using Weights and Biases logging for deep learning applications on NERSC machines☆11Updated last year
- ALCF Computational Performance Workshop☆37Updated 2 years ago
- COCCL: Compression and precision co-aware collective communication library☆22Updated last month
- The ALCF hosts a regular simulation, data, and learning workshop to help users scale their applications. This repository contains the exa…☆63Updated 5 months ago
- A parallel framework for training deep neural networks☆58Updated last month
- CUDA training materials☆19Updated 8 months ago
- A C++based implementation of the TeaLeaf heat conduction mini-app. This implementation of TeaLeaf replicates the functionality of the ref…☆23Updated 8 months ago
- "wow, that is really fast." - Kyle Gerard Felker☆9Updated 3 years ago
- Material for the SC21 Deep Learning at Scale Tutorial☆25Updated 2 years ago
- Train across all your devices, ezpz 🍋☆20Updated this week
- SC23 Deep Learning at Scale Tutorial Material☆43Updated 7 months ago
- Example applications for the Department of Energy Computational Science Graduate Fellowship☆16Updated 9 months ago
- This is the public repo for the MLPerf DeepCAM climate data segmentation proposal.☆16Updated 3 years ago
- How to use node-local MPI rank IDs to manually map MPI ranks to GPUs☆14Updated 5 years ago
- A hands-on introduction to tuning GPU kernels using Kernel Tuner https://github.com/KernelTuner/kernel_tuner/☆30Updated 2 weeks ago
- ☆14Updated last year
- Python Interface to the Data and Learning Hub for Science☆28Updated last year
- Specialized Parallel Linear Algebra, providing distributed GEMM functionality for specific matrix distributions with optional GPU acceler…☆29Updated 10 months ago
- CPU and GPU tutorial examples☆13Updated 3 weeks ago
- The JUBE benchmarking environment provides a script based framework to easily create benchmark sets, run those sets on different computer…☆37Updated 10 months ago