argonne-lcf / Megatron-DeepSpeedLinks
Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆16Updated this week
Alternatives and similar repositories for Megatron-DeepSpeed
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below
Sorting:
- Material for the SC22 Deep Learning at Scale Tutorial☆41Updated 2 years ago
- This is a repository with examples to run inference endpoints on various ALCF clusters☆24Updated 3 weeks ago
- SC24 Deep Learning at Scale Tutorial Material☆33Updated 6 months ago
- Cosmic Tagging Network for Neutrino Physics☆13Updated last year
- ☆43Updated last month
- ☆21Updated 4 years ago
- Collection of small examples for running on ALCF resources☆19Updated last month
- ☆53Updated this week
- Guidelines on using Weights and Biases logging for deep learning applications on NERSC machines☆13Updated 2 years ago
- Sparsity support for PyTorch☆36Updated 5 months ago
- ALCF Computational Performance Workshop☆37Updated 2 years ago
- A parallel framework for training deep neural networks☆63Updated 5 months ago
- SC23 Deep Learning at Scale Tutorial Material☆47Updated 11 months ago
- COCCL: Compression and precision co-aware collective communication library☆24Updated 5 months ago
- ☆50Updated 3 months ago
- Train across all your devices, ezpz 🍋☆23Updated this week
- A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.☆271Updated this week
- Reference implementations of MLPerf™ HPC training benchmarks☆49Updated 6 months ago
- AI Training Series Material☆37Updated 11 months ago
- Minimal yet performant LLM examples in pure JAX☆150Updated this week
- Benchmark implementation of CosmoFlow in TensorFlow Keras☆21Updated last year
- Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…☆62Updated 2 months ago
- ☆172Updated last year
- extensible collectives library in triton☆88Updated 4 months ago
- ☆232Updated last week
- ☆132Updated last month
- Effective transpose on Hopper GPU☆23Updated 3 months ago
- A hands-on introduction to tuning GPU kernels using Kernel Tuner https://github.com/KernelTuner/kernel_tuner/☆32Updated 4 months ago
- Repository of machine learning benchmarks☆40Updated last week
- Small scale distributed training of sequential deep learning models, built on Numpy and MPI.☆136Updated last year