argonne-lcf / Megatron-DeepSpeedLinks
Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆17Updated 2 months ago
Alternatives and similar repositories for Megatron-DeepSpeed
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below
Sorting:
- Material for the SC22 Deep Learning at Scale Tutorial☆41Updated 2 years ago
- Collection of small examples for running on ALCF resources☆21Updated 3 weeks ago
- This is a repository with examples to run inference endpoints on various ALCF clusters☆27Updated 2 months ago
- Cosmic Tagging Network for Neutrino Physics☆13Updated last year
- ☆49Updated 6 months ago
- SC24 Deep Learning at Scale Tutorial Material☆33Updated 11 months ago
- Train across all your devices, ezpz 🍋☆26Updated last week
- Sparsity support for PyTorch☆38Updated 10 months ago
- Tokamax: A GPU and TPU kernel library.☆165Updated last week
- COCCL: Compression and precision co-aware collective communication library☆29Updated 10 months ago
- Guidelines on using Weights and Biases logging for deep learning applications on NERSC machines☆13Updated 2 years ago
- ALCF Computational Performance Workshop☆38Updated 3 years ago
- ☆55Updated 2 months ago
- ☆137Updated 3 months ago
- A hands-on introduction to tuning GPU kernels using Kernel Tuner https://github.com/KernelTuner/kernel_tuner/☆36Updated 3 months ago
- Parallel framework for training and fine-tuning deep neural networks☆70Updated 2 months ago
- ☆20Updated 6 years ago
- A PyTorch native platform for training generative AI models☆14Updated 2 months ago
- AI Training Series Material☆40Updated 3 months ago
- Benchmark implementation of CosmoFlow in TensorFlow Keras☆22Updated last year
- SC23 Deep Learning at Scale Tutorial Material☆49Updated last year
- This repository contains companion software for the Colfax Research paper "Categorical Foundations for CuTe Layouts".☆95Updated 4 months ago
- Anatomy of High-Performance GEMM with Online Fault Tolerance on GPUs☆13Updated 9 months ago
- ☆39Updated last month
- Minimal yet performant LLM examples in pure JAX☆233Updated 2 weeks ago
- torchax is a PyTorch frontend for JAX. It gives JAX the ability to author JAX programs using familiar PyTorch syntax. It also provides JA…☆171Updated this week
- Evaluating Large Language Models for CUDA Code Generation ComputeEval is a framework designed to generate and evaluate CUDA code from Lar…☆94Updated 3 weeks ago
- Reference implementations of MLPerf™ HPC training benchmarks☆49Updated 11 months ago
- Write a fast kernel and run it on Discord. See how you compare against the best!☆68Updated this week
- This is repository for a I/O benchmark which represents Scientific Deep Learning Workloads.☆23Updated 3 years ago