kimihe / Octo
Create tiny ML systems for on-device learning.
☆20Updated 3 years ago
Alternatives and similar repositories for Octo:
Users that are interested in Octo are comparing it to the libraries listed below
- Partial implementation of paper "DEEP GRADIENT COMPRESSION: REDUCING THE COMMUNICATION BANDWIDTH FOR DISTRIBUTED TRAINING"☆31Updated 4 years ago
- Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k c…☆25Updated 2 years ago
- Federated Dynamic Sparse Training☆30Updated 2 years ago
- A curated list of early exiting (LLM, CV, NLP, etc)☆49Updated 8 months ago
- ☆19Updated 2 years ago
- A Cluster-Wide Model Manager to Accelerate DNN Training via Automated Training Warmup☆34Updated 2 years ago
- [ICDCS 2023] Evaluation and Optimization of Gradient Compression for Distributed Deep Learning☆10Updated 2 years ago
- AN EFFICIENT AND GENERAL FRAMEWORK FOR LAYERWISE-ADAPTIVE GRADIENT COMPRESSION☆13Updated last year
- ☆10Updated 3 years ago
- ☆14Updated 3 years ago
- Measuring and predicting on-device metrics (latency, power, etc.) of machine learning models☆66Updated 2 years ago
- Understanding Top-k Sparsification in Distributed Deep Learning☆24Updated 5 years ago
- Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples [NeurIPS 2021]☆31Updated 3 years ago
- Code for "Adaptive Gradient Quantization for Data-Parallel SGD", published in NeurIPS 2020.☆30Updated 4 years ago
- Changing several bit which overwhelms the quantized CNN☆42Updated 5 years ago
- Source code for the paper: "A Latency-Predictable Multi-Dimensional Optimization Framework forDNN-driven Autonomous Systems"☆22Updated 4 years ago
- [Neurips 2021] Sparse Training via Boosting Pruning Plasticity with Neuroregeneration☆31Updated 2 years ago
- [ICML 2021] "Double-Win Quant: Aggressively Winning Robustness of Quantized DeepNeural Networks via Random Precision Training and Inferen…☆13Updated 3 years ago
- ☆10Updated 3 years ago
- Official implementation for ECCV 2022 paper LIMPQ - "Mixed-Precision Neural Network Quantization via Learned Layer-wise Importance"☆54Updated 2 years ago
- We present a set of all-reduce compatible gradient compression algorithms which significantly reduce the communication overhead while mai…☆9Updated 3 years ago
- Post-training sparsity-aware quantization☆34Updated 2 years ago
- Dual-way gradient sparsification approach for async DNN training, based on PyTorch.☆11Updated 2 years ago
- PipeTransformer: Automated Elastic Pipelining for Distributed Training of Large-scale Models. ICML 2021☆56Updated 3 years ago
- ☆99Updated last year
- Pytorch-based early exit network inspired by branchynet☆31Updated 3 weeks ago
- Implementation of (overlap) local SGD in Pytorch☆33Updated 4 years ago
- [ICML 2021] "Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training" by Shiwei Liu, Lu Yin, De…☆45Updated last year
- GRACE - GRAdient ComprEssion for distributed deep learning☆139Updated 9 months ago
- Any-Precision Deep Neural Networks (AAAI 2021)☆60Updated 5 years ago