argonne-lcf / Megatron-DeepSpeedLinks
Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆17Updated last week
Alternatives and similar repositories for Megatron-DeepSpeed
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below
Sorting:
- Material for the SC22 Deep Learning at Scale Tutorial☆41Updated 2 years ago
- SC24 Deep Learning at Scale Tutorial Material☆33Updated 9 months ago
- Cosmic Tagging Network for Neutrino Physics☆13Updated last year
- This is a repository with examples to run inference endpoints on various ALCF clusters☆26Updated last week
- Collection of small examples for running on ALCF resources☆19Updated 3 months ago
- ☆47Updated 4 months ago
- Benchmark implementation of CosmoFlow in TensorFlow Keras☆21Updated last year
- Sparsity support for PyTorch☆37Updated 7 months ago
- ☆21Updated 5 years ago
- ALCF Computational Performance Workshop☆38Updated 3 years ago
- AI Training Series Material☆38Updated last month
- Guidelines on using Weights and Biases logging for deep learning applications on NERSC machines☆13Updated 2 years ago
- ☆28Updated 10 months ago
- ☆51Updated last week
- Parallel framework for training and fine-tuning deep neural networks☆68Updated last week
- Train across all your devices, ezpz 🍋☆24Updated last week
- A hands-on introduction to tuning GPU kernels using Kernel Tuner https://github.com/KernelTuner/kernel_tuner/☆35Updated 3 weeks ago
- COCCL: Compression and precision co-aware collective communication library☆27Updated 8 months ago
- SC23 Deep Learning at Scale Tutorial Material☆48Updated last year
- Anatomy of High-Performance GEMM with Online Fault Tolerance on GPUs☆12Updated 7 months ago
- Reference implementations of MLPerf™ HPC training benchmarks☆49Updated 8 months ago
- ☆134Updated last month
- Sources for the Oak Ridge Leadership Computing Facility User Documentation☆67Updated last week
- single-GPU to multi-GPU training of PyTorch apps at NERSC☆22Updated last year
- ☆61Updated this week
- ☆16Updated last year
- Tokamax: A GPU and TPU kernel library.☆106Updated this week
- torchax is a PyTorch frontend for JAX. It gives JAX the ability to author JAX programs using familiar PyTorch syntax. It also provides JA…☆122Updated this week
- Material for the SC21 Deep Learning at Scale Tutorial☆27Updated 2 years ago
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆150Updated 2 years ago