argonne-lcf / Megatron-DeepSpeedLinks
Ongoing research training transformer language models at scale, including: BERT & GPT-2
☆16Updated last week
Alternatives and similar repositories for Megatron-DeepSpeed
Users that are interested in Megatron-DeepSpeed are comparing it to the libraries listed below
Sorting:
- Material for the SC22 Deep Learning at Scale Tutorial☆41Updated 2 years ago
- This is a repository with examples to run inference endpoints on various ALCF clusters☆26Updated 2 weeks ago
- SC24 Deep Learning at Scale Tutorial Material☆33Updated 8 months ago
- Cosmic Tagging Network for Neutrino Physics☆13Updated last year
- Collection of small examples for running on ALCF resources☆19Updated 2 months ago
- Benchmark implementation of CosmoFlow in TensorFlow Keras☆21Updated last year
- Sparsity support for PyTorch☆37Updated 6 months ago
- COCCL: Compression and precision co-aware collective communication library☆24Updated 6 months ago
- LLM checkpointing for DeepSpeed/Megatron☆21Updated this week
- Train across all your devices, ezpz 🍋☆24Updated this week
- Parallel framework for training and fine-tuning deep neural networks☆64Updated 6 months ago
- ☆51Updated 4 months ago
- ☆21Updated 4 years ago
- ☆46Updated 2 months ago
- Guidelines on using Weights and Biases logging for deep learning applications on NERSC machines☆13Updated 2 years ago
- ALCF Computational Performance Workshop☆38Updated 3 years ago
- Anatomy of High-Performance GEMM with Online Fault Tolerance on GPUs☆12Updated 6 months ago
- SC23 Deep Learning at Scale Tutorial Material☆47Updated last year
- ☆57Updated last week
- Reference implementations of MLPerf™ HPC training benchmarks☆49Updated 7 months ago
- This repository contains companion software for the Colfax Research paper "Categorical Foundations for CuTe Layouts".☆57Updated 2 weeks ago
- ☆21Updated 2 months ago
- ☆14Updated 2 years ago
- Evaluating Large Language Models for CUDA Code Generation ComputeEval is a framework designed to generate and evaluate CUDA code from Lar…☆66Updated last week
- Intel® Extension for DeepSpeed* is an extension to DeepSpeed that brings feature support with SYCL kernels on Intel GPU(XPU) device. Note…☆63Updated 3 months ago
- Hand-Rolled GPU communications library☆37Updated this week
- General Matrix Multiplication using NVIDIA Tensor Cores☆21Updated 8 months ago
- A Python-embedded DSL that makes it easy to write fast, scalable ML kernels with minimal boilerplate.☆328Updated this week
- Personal solutions to the Triton Puzzles☆20Updated last year
- JaxPP is a library for JAX that enables flexible MPMD pipeline parallelism for large-scale LLM training☆54Updated last week