vineeths96 / Gradient-CompressionLinks
We present a set of all-reduce compatible gradient compression algorithms which significantly reduce the communication overhead while maintaining the performance of vanilla SGD. We empirically evaluate the performance of the compression methods by training deep neural networks on the CIFAR10 dataset.
☆10Updated 3 years ago
Alternatives and similar repositories for Gradient-Compression
Users that are interested in Gradient-Compression are comparing it to the libraries listed below
Sorting:
- Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k c…☆26Updated 2 years ago
- A Cluster-Wide Model Manager to Accelerate DNN Training via Automated Training Warmup☆35Updated 2 years ago
- [ICDCS 2023] DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining☆12Updated last year
- ☆69Updated 2 years ago
- Hi-Speed DNN Training with Espresso: Unleashing the Full Potential of Gradient Compression with Near-Optimal Usage Strategies (EuroSys '2…☆15Updated last year
- ☆25Updated last year
- [ICDCS 2023] Evaluation and Optimization of Gradient Compression for Distributed Deep Learning☆10Updated 2 years ago
- [ACM SoCC'22] Pisces: Efficient Federated Learning via Guided Asynchronous Training☆12Updated 2 months ago
- This is a list of awesome edgeAI inference related papers.☆96Updated last year
- Official resporitory for "IPDPS' 24 QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices".☆20Updated last year
- AN EFFICIENT AND GENERAL FRAMEWORK FOR LAYERWISE-ADAPTIVE GRADIENT COMPRESSION☆15Updated last year
- PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021☆56Updated 3 years ago
- [ICLR 2018] Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training☆222Updated last year
- A Sparse-tensor Communication Framework for Distributed Deep Learning☆13Updated 3 years ago
- ☆10Updated 4 years ago
- ☆14Updated 3 years ago
- [IJCAI2023] An automated parallel training system that combines the advantages from both data and model parallelism. If you have any inte…☆51Updated 2 years ago
- Partial implementation of paper "DEEP GRADIENT COMPRESSION: REDUCING THE COMMUNICATION BANDWIDTH FOR DISTRIBUTED TRAINING"☆31Updated 4 years ago
- THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression☆19Updated 11 months ago
- Create tiny ML systems for on-device learning.☆20Updated 4 years ago
- Official Repo for "LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization"☆34Updated last week
- Code for reproducing experiments performed for Accoridon☆13Updated 4 years ago
- ddl-benchmarks: Benchmarks for Distributed Deep Learning☆37Updated 5 years ago
- MobiSys#114☆21Updated last year
- A computation-parallel deep learning architecture.☆13Updated 5 years ago
- PipeEdge: Pipeline Parallelism for Large-Scale Model Inference on Heterogeneous Edge Devices☆35Updated last year
- Layer-wise Sparsification of Distributed Deep Learning☆10Updated 5 years ago
- Understanding Top-k Sparsification in Distributed Deep Learning☆24Updated 5 years ago
- ☆15Updated 4 years ago
- GRACE - GRAdient ComprEssion for distributed deep learning☆140Updated 11 months ago