argonne-lcf / Megatron-DeepSpeedLinks
Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆17Updated 3 weeks ago
Alternatives and similar repositories for Megatron-DeepSpeed
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below
Sorting:
- Cosmic Tagging Network for Neutrino Physics☆13Updated last year
- Material for the SC22 Deep Learning at Scale Tutorial☆41Updated 2 years ago
- Sparsity support for PyTorch☆37Updated 8 months ago
- This is a repository with examples to run inference endpoints on various ALCF clusters☆26Updated 3 weeks ago
- ALCF Computational Performance Workshop☆38Updated 3 years ago
- SC24 Deep Learning at Scale Tutorial Material☆33Updated 10 months ago
- Benchmark implementation of CosmoFlow in TensorFlow Keras☆22Updated last year
- ☆20Updated 6 years ago
- ☆21Updated 5 years ago
- MLPerf™ logging library☆37Updated this week
- COCCL: Compression and precision co-aware collective communication library☆29Updated 8 months ago
- AI Training Series Material☆38Updated 2 months ago
- ☆47Updated 4 months ago
- This repository contains companion software for the Colfax Research paper "Categorical Foundations for CuTe Layouts".☆82Updated 2 months ago
- ☆52Updated 3 weeks ago
- Collection of small examples for running on ALCF resources☆19Updated 4 months ago
- Anatomy of High-Performance GEMM with Online Fault Tolerance on GPUs☆13Updated 8 months ago
- ☆28Updated 10 months ago
- Test suite for probing the numerical behavior of NVIDIA tensor cores☆41Updated last year
- This is the public repo for the MLPerf DeepCAM climate data segmentation proposal.☆16Updated 2 months ago
- ☆67Updated this week
- AMD RAD's multi-GPU Triton-based framework for seamless multi-GPU programming☆121Updated this week
- Train across all your devices, ezpz 🍋☆24Updated 2 weeks ago
- Reference implementations of MLPerf™ HPC training benchmarks☆49Updated 9 months ago
- extensible collectives library in triton☆91Updated 8 months ago
- Personal solutions to the Triton Puzzles☆20Updated last year
- LLM checkpointing for DeepSpeed/Megatron☆21Updated last week
- A hands-on introduction to tuning GPU kernels using Kernel Tuner https://github.com/KernelTuner/kernel_tuner/☆35Updated last month
- Parallel framework for training and fine-tuning deep neural networks☆70Updated last month
- ☆12Updated 3 months ago