It is implementation of Research paper "DEEP GRADIENT COMPRESSION: REDUCING THE COMMUNICATION BANDWIDTH FOR DISTRIBUTED TRAINING". Deep gradient compression is a technique by which the gradients are compressed before they are being sent. This approach greatly reduces the communication bandwidth and thus improves multi node training.
☆18Aug 14, 2019Updated 6 years ago
Alternatives and similar repositories for DeepGradientCompression
Users that are interested in DeepGradientCompression are comparing it to the libraries listed below
Sorting:
- Partial implementation of paper "DEEP GRADIENT COMPRESSION: REDUCING THE COMMUNICATION BANDWIDTH FOR DISTRIBUTED TRAINING"☆32Nov 20, 2020Updated 5 years ago
- A computation-parallel deep learning architecture.☆13Sep 25, 2019Updated 6 years ago
- Atomo: Communication-efficient Learning via Atomic Sparsification☆28Dec 9, 2018Updated 7 years ago
- vector quantization for stochastic gradient descent.☆35May 12, 2020Updated 5 years ago
- [ICLR 2018] Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training☆226Jul 10, 2024Updated last year
- GRACE - GRAdient ComprEssion for distributed deep learning☆139Jul 23, 2024Updated last year
- Practical low-rank gradient compression for distributed optimization: https://arxiv.org/abs/1905.13727☆149Oct 29, 2024Updated last year
- CMFL: Mitigating Communication Overhead for Federated Learning / PyTorch reimplementation.☆29Oct 18, 2019Updated 6 years ago
- SGD with compressed gradients and error-feedback: https://arxiv.org/abs/1901.09847☆32Jul 25, 2024Updated last year
- ☆33Dec 3, 2019Updated 6 years ago
- Edge Impulse FOMO Implementation from scratch☆16Updated this week
- SKFAC Preconditioner for MindSpore☆12Jul 2, 2021Updated 4 years ago
- CoCoFL: Communication- and Computation-Aware Federated Learning via Partial NN Freezing and Quantization☆13Aug 3, 2024Updated last year
- ☆18Nov 1, 2025Updated 3 months ago
- [ICDCS 2023] Evaluation and Optimization of Gradient Compression for Distributed Deep Learning☆10Apr 28, 2023Updated 2 years ago
- FastTrack4LLM 是一个为大模型学习者准备的大模型学习与实践框架，帮助他们轻松掌握大模型的核心原理与训练流程，让每个人都能真正理解大模型的内部机制。本项目不仅完整复现了 LLaMA、Qwen、DeepSeek 等主流开源大模型架构，还覆盖了大模型的全生命周期：To…☆24Nov 6, 2025Updated 3 months ago
- Code for the paper "Semantic Communication Empowered Physical-layer Network Coding"☆10Sep 6, 2022Updated 3 years ago
- Generalized Method of Moments estimation☆13Mar 23, 2025Updated 11 months ago
- The official implementation of paper "Overcoming Data and Model heterogeneities in Decentralized Federated Learning via Synthetic Anchors…☆14Jun 14, 2024Updated last year
- Communication-Efficient Feature-based Federated Transfer Learning☆14Jun 20, 2024Updated last year
- VHDL Implementation☆13Oct 9, 2014Updated 11 years ago
- Experiments codes for WSDM '24 paper "MultiFS: Automated Multi-Scenario Feature Selection in Deep Recommender Systems"☆11May 31, 2024Updated last year
- Solving Logic Grid Puzzles with Part-of-Speech Tagging and First-Order Logic☆11Dec 18, 2016Updated 9 years ago
- Effective Attention Sheds Light On Interpretability - Findings of ACL2021☆11May 16, 2021Updated 4 years ago
- Unofficial implementation of the Ask-LLM paper 'How to Train Data-Efficient LLMs', arXiv:2402.09668.☆12Jun 19, 2024Updated last year
- A light-weight encryption & decryption library, Built with power and simplicity!☆11Oct 1, 2023Updated 2 years ago
- Document of WeCross https://wecross.readthedocs.io/zh_CN/latest/☆10Apr 8, 2024Updated last year
- 本次实训按照指导书要求，设计并实现了一个简单的图像检索系统，实现了从颜色特征、纹理特征、形状特征三个方面对图像进行匹配。其中颜色特征是提取自RGB颜色空间的颜色矩；纹理特征是来自图像0度、45度、90度、135度四个角度的灰度共生矩阵的纹理一致性、纹理对比度、纹理熵，纹理相…☆12Feb 15, 2022Updated 4 years ago
- Three implementations of FedAvg: numpy, pytorch and tensorflow federated.☆45Jun 28, 2022Updated 3 years ago
- Based in my last research paper - 2021☆10Dec 9, 2023Updated 2 years ago
- ☆13Jul 23, 2023Updated 2 years ago
- ☆14Oct 12, 2024Updated last year
- Official implementation of DapperFL.☆13Oct 29, 2024Updated last year
- In this project, we build a simple federated deep learning model (i.e., LeNet model) that classifies 62 differents Belgian Traffic Signs.☆14Feb 23, 2021Updated 5 years ago
- Speech recognition with federated learning☆11Jan 9, 2020Updated 6 years ago
- The official implementation of "Federated Learning with Label-Masking Distillation"☆11Oct 28, 2023Updated 2 years ago
- ☆13Oct 29, 2022Updated 3 years ago
- Implementation for ACProp ( Momentum centering and asynchronous update for adaptive gradient methdos, NeurIPS 2021)☆16Oct 11, 2021Updated 4 years ago
- Elgamal Encryption using Elliptic Curve Cryptography in Python 2.7☆13Nov 17, 2018Updated 7 years ago