vineeths96 / Gradient-CompressionLinks
We present a set of all-reduce compatible gradient compression algorithms which significantly reduce the communication overhead while maintaining the performance of vanilla SGD. We empirically evaluate the performance of the compression methods by training deep neural networks on the CIFAR10 dataset.
☆10Updated 4 years ago
Alternatives and similar repositories for Gradient-Compression
Users that are interested in Gradient-Compression are comparing it to the libraries listed below
Sorting:
- Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k c…☆27Updated 2 years ago
- A Sparse-tensor Communication Framework for Distributed Deep Learning☆13Updated 4 years ago
- [ICDCS 2023] DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining☆11Updated last year
- [ICDCS 2023] Evaluation and Optimization of Gradient Compression for Distributed Deep Learning☆10Updated 2 years ago
- Hi-Speed DNN Training with Espresso: Unleashing the Full Potential of Gradient Compression with Near-Optimal Usage Strategies (EuroSys '2…☆15Updated 2 years ago
- ☆14Updated 3 years ago
- This is a list of awesome edgeAI inference related papers.☆98Updated last year
- A Cluster-Wide Model Manager to Accelerate DNN Training via Automated Training Warmup☆35Updated 2 years ago
- PyTorch compilation tutorial covering TorchScript, torch.fx, and Slapo☆17Updated 2 years ago
- ☆21Updated 3 years ago
- ☆15Updated 4 years ago
- ☆10Updated 4 years ago
- ☆25Updated 2 years ago
- Code for reproducing experiments performed for Accoridon☆13Updated 4 years ago
- Dual-way gradient sparsification approach for async DNN training, based on PyTorch.☆11Updated 2 years ago
- MobiSys#114☆22Updated 2 years ago
- [ACM SoCC'22] Pisces: Efficient Federated Learning via Guided Asynchronous Training☆13Updated 6 months ago
- This repository is the official implementation of 'EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Lea…☆14Updated 3 years ago
- Paper list for accleration of transformers☆13Updated 2 years ago
- Create tiny ML systems for on-device learning.☆20Updated 4 years ago
- ☆68Updated 2 years ago
- Paper list of federated learning: About system design☆13Updated 3 years ago
- Official resporitory for "IPDPS' 24 QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices".☆20Updated last year
- Source code for the paper: "A Latency-Predictable Multi-Dimensional Optimization Framework forDNN-driven Autonomous Systems"☆22Updated 4 years ago
- THC: Accelerating Distributed Deep Learning Using Tensor Homomorphic Compression☆20Updated last year
- AN EFFICIENT AND GENERAL FRAMEWORK FOR LAYERWISE-ADAPTIVE GRADIENT COMPRESSION☆14Updated 2 years ago
- ☆29Updated last month
- Understanding Top-k Sparsification in Distributed Deep Learning☆24Updated 6 years ago
- Layer-wise Sparsification of Distributed Deep Learning☆10Updated 5 years ago
- GRACE - GRAdient ComprEssion for distributed deep learning☆139Updated last year