We present a set of all-reduce compatible gradient compression algorithms which significantly reduce the communication overhead while maintaining the performance of vanilla SGD. We empirically evaluate the performance of the compression methods by training deep neural networks on the CIFAR10 dataset.
☆10Nov 14, 2021Updated 4 years ago
Alternatives and similar repositories for Gradient-Compression
Users that are interested in Gradient-Compression are comparing it to the libraries listed below
Sorting:
- ☆33Dec 3, 2019Updated 6 years ago
- [ICDCS 2023] Evaluation and Optimization of Gradient Compression for Distributed Deep Learning☆10Apr 28, 2023Updated 2 years ago
- FedShare: Secure Aggregation based on Additive Secret Sharing in Federated Learning☆21Jan 15, 2026Updated 2 months ago
- A Sparse-tensor Communication Framework for Distributed Deep Learning☆13Nov 1, 2021Updated 4 years ago
- ☆10Jun 4, 2021Updated 4 years ago
- Code for reproducing experiments performed for Accoridon☆13Jun 11, 2021Updated 4 years ago
- Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k c…☆27Dec 10, 2022Updated 3 years ago
- Code for "Adaptive Gradient Quantization for Data-Parallel SGD", published in NeurIPS 2020.☆30Jan 14, 2021Updated 5 years ago
- Q-RR, DIANA-RR, Q-NASTYA, NASTYA-DIANA, QSGD, DIANA, FedCOM and FedPAQ on logistic loss with L2 regularization☆11Nov 1, 2022Updated 3 years ago
- Implementation of EF and EF21 methods and their comparison on several tasks☆10Feb 24, 2024Updated 2 years ago
- vector quantization for stochastic gradient descent.☆35May 12, 2020Updated 5 years ago
- GRACE - GRAdient ComprEssion for distributed deep learning☆139Jul 23, 2024Updated last year
- The source code of the paper "Efficient Privacy-Preserving Federated Learning with Compressed Sensing"☆22May 23, 2024Updated last year
- Efficient Secure Aggregation for Privacy-Preserving Federated Machine Learning☆15Aug 21, 2025Updated 6 months ago
- ☆10Jun 21, 2024Updated last year
- ☆17May 10, 2024Updated last year
- SGQuant: Squeezing the Last Bit on Graph Neural Networks with Specialized Quantization☆11Aug 12, 2020Updated 5 years ago
- Hi-Speed DNN Training with Espresso: Unleashing the Full Potential of Gradient Compression with Near-Optimal Usage Strategies (EuroSys '2…☆15Sep 21, 2023Updated 2 years ago
- Understanding Top-k Sparsification in Distributed Deep Learning☆24Nov 15, 2019Updated 6 years ago
- ☆15Jul 13, 2021Updated 4 years ago
- GPU scheduler for elastic/distributed deep learning workloads in Kubernetes cluster (IC2E'23)☆34Nov 11, 2023Updated 2 years ago
- Aggregation Service for Federated Learning: An Efficient, Secure, and More Resilient Realization☆13Jun 21, 2021Updated 4 years ago
- CIFAR-10 image classification of imbalanced data using bottleneck features extracted from the autoencoder.☆10Sep 26, 2019Updated 6 years ago
- Accepted at MLNCP NeurIPS 2024: QFed+FHE: Quantum Federated Learning with Secure Fully Homomorphic Encryption (FHE)☆15Dec 27, 2024Updated last year
- [RAL 2024] MACIM: Multi-Agent Collaborative Implicit Mapping Through Distributed Optimizationn☆15Sep 25, 2023Updated 2 years ago
- ☆21Sep 20, 2022Updated 3 years ago
- Automatic ReLU Reduction☆15Dec 20, 2023Updated 2 years ago
- Artifacts of VLDB'22 paper "COMET: A Novel Memory-Efficient Deep Learning TrainingFramework by Using Error-Bounded Lossy Compression"☆10Aug 2, 2022Updated 3 years ago
- Real-Time Sparse Visual Odometry with Neural Radiance Fields☆17Aug 11, 2024Updated last year
- ☆12Aug 22, 2023Updated 2 years ago
- Code for paper "Learning a Code: Machine Learning for Approximate Non-Linear Coded-Computation"☆11Dec 21, 2020Updated 5 years ago
- 这是一个使用vue配合element-ui开发的一个电商后台管理系统项目，项目都都是使用的前端最新技术☆13Jan 4, 2023Updated 3 years ago
- SC 2021, "LogECMem: Coupling Erasure-Coded In-Memory Key-Value Stores with Parity Logging"☆12Jul 12, 2021Updated 4 years ago
- An attempt to replicate the paper "Multi-shot Pedestrian Re-identification via Sequential Decision Making (CVPR2018)"☆10Nov 16, 2019Updated 6 years ago
- [ICASSP 2024] VGDiffZero: Text-to-image Diffusion Models Can Be Zero-shot Visual Grounders☆17Feb 11, 2025Updated last year
- Proof of Concept of a Federated Learning framework that maintains the privacy of the participants involved.☆14Feb 4, 2026Updated last month
- Natural Language Processing models using private and secure data. Powered by OpenMined's tools PySyft and SyferText.☆11Feb 11, 2021Updated 5 years ago
- Communication-Efficient Federated Learning through Adaptive Weight Clustering and Server-Side Distillation☆24May 22, 2024Updated last year
- PyFed generic framework of benchmark for Federated Learning☆11Oct 9, 2021Updated 4 years ago