argonne-lcf / Megatron-DeepSpeedLinks
Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆17Updated last month
Alternatives and similar repositories for Megatron-DeepSpeed
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below
Sorting:
- SC24 Deep Learning at Scale Tutorial Material☆32Updated 4 months ago
- Cosmic Tagging Network for Neutrino Physics☆13Updated 11 months ago
- This is a repository with examples to run inference endpoints on various ALCF clusters☆20Updated last week
- CPU and GPU tutorial examples☆13Updated 2 months ago
- Material for the SC22 Deep Learning at Scale Tutorial☆41Updated last year
- Benchmark implementation of CosmoFlow in TensorFlow Keras☆21Updated last year
- Pragmatic, Productive, and Portable Affinity for HPC☆38Updated 3 weeks ago
- ☆36Updated last month
- Collection of small examples for running on ALCF resources☆19Updated last week
- COCCL: Compression and precision co-aware collective communication library☆22Updated 2 months ago
- Sparsity support for PyTorch☆35Updated 2 months ago
- Reference implementations of MLPerf™ HPC training benchmarks☆48Updated 3 months ago
- JUPITER Benchmark Suite☆16Updated 10 months ago
- ☆21Updated 4 years ago
- This is the public repo for the MLPerf DeepCAM climate data segmentation proposal.☆16Updated 3 years ago
- Graph-indexed Pandas DataFrames for analyzing hierarchical performance data☆33Updated 3 weeks ago
- A parallel framework for training deep neural networks☆60Updated 2 months ago
- ☆18Updated 5 years ago
- Specialized Parallel Linear Algebra, providing distributed GEMM functionality for specific matrix distributions with optional GPU acceler…☆29Updated 11 months ago
- AI Training Series Material☆35Updated 8 months ago
- A hands-on introduction to tuning GPU kernels using Kernel Tuner https://github.com/KernelTuner/kernel_tuner/☆31Updated last month
- The ALCF hosts a regular simulation, data, and learning workshop to help users scale their applications. This repository contains the exa…☆63Updated 7 months ago
- ALCF Computational Performance Workshop☆37Updated 2 years ago
- Benchmarks☆17Updated last month
- A hierarchical collective communications library with portable optimizations☆35Updated 5 months ago
- AMD HPC Research Fund Cloud☆13Updated 3 weeks ago
- This material contains content on how to profile and optimize simple Pytorch mnist code using NVIDIA Nsight Systems and Pytorch Profiler☆12Updated 2 years ago
- Analyze parallel execution traces using pandas dataframes☆22Updated last month
- Guidelines on using Weights and Biases logging for deep learning applications on NERSC machines☆13Updated last year
- NPBench - A Benchmarking Suite for High-Performance NumPy☆81Updated 3 weeks ago