hpcaitech / ColossalAI-BenchmarkLinks
Performance benchmarking with ColossalAI
☆39Updated 2 years ago
Alternatives and similar repositories for ColossalAI-Benchmark
Users that are interested in ColossalAI-Benchmark are comparing it to the libraries listed below
Sorting:
- Scalable PaLM implementation of PyTorch☆190Updated 2 years ago
- A Python library transfers PyTorch tensors between CPU and NVMe☆116Updated 7 months ago
- ☆27Updated 3 years ago
- ☆105Updated 10 months ago
- ☆74Updated 4 years ago
- Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.☆67Updated 3 months ago
- Automated Parallelization System and Infrastructure for Multiple Ecosystems☆79Updated 7 months ago
- PyTorch bindings for CUTLASS grouped GEMM.☆127Updated 5 months ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆210Updated 10 months ago
- ☆96Updated 9 months ago
- ☆80Updated last month
- PyTorch bindings for CUTLASS grouped GEMM.☆100Updated 3 weeks ago
- ☆42Updated 2 years ago
- [IJCAI2023] An automated parallel training system that combines the advantages from both data and model parallelism. If you have any inte…☆51Updated 2 years ago
- [USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Paral…☆55Updated 10 months ago
- ☆148Updated 11 months ago
- An efficient GPU support for LLM inference with x-bit quantization (e.g. FP6,FP5).☆252Updated 7 months ago
- ☆119Updated last year
- (NeurIPS 2022) Automatically finding good model-parallel strategies, especially for complex models and clusters.☆39Updated 2 years ago
- SparseTIR: Sparse Tensor Compiler for Deep Learning☆138Updated 2 years ago
- Sky Computing: Accelerating Geo-distributed Computing in Federated Learning☆91Updated 2 years ago
- nnScaler: Compiling DNN models for Parallel Training☆113Updated last week
- Odysseus: Playground of LLM Sequence Parallelism☆70Updated last year
- ATC23 AE☆45Updated 2 years ago
- 📑 Dive into Big Model Training☆114Updated 2 years ago
- A high-performance distributed deep learning system targeting large-scale and automated distributed training. If you have any interests, …☆112Updated last year
- A collection of memory efficient attention operators implemented in the Triton language.☆272Updated last year
- A simple calculation for LLM MFU.☆38Updated 3 months ago
- Memory Optimizations for Deep Learning (ICML 2023)☆64Updated last year
- Examples of training models with hybrid parallelism using ColossalAI☆340Updated 2 years ago