mryab / learning-at-home
"Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts" (NeurIPS 2020), original PyTorch implementation
☆53Updated 4 years ago
Related projects ⓘ
Alternatives and complementary repositories for learning-at-home
- Code for the paper "Secure Distributed Training at Scale" (ICML 2022)☆14Updated 2 years ago
- A GPT, made only of MLPs, in Jax☆55Updated 3 years ago
- Memory-efficient transformer. Work in progress.☆19Updated 2 years ago
- Python library for argument and configuration management☆53Updated last year
- A centralized place for deep thinking code and experiments☆76Updated last year
- ☆34Updated last year
- Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing☆47Updated 2 years ago
- Pretrained TorchVision models on CIFAR10 dataset (with weights)☆24Updated 4 years ago
- Latest Weight Averaging (NeurIPS HITY 2022)☆20Updated last year
- Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper☆78Updated 3 years ago
- CIFAR-5m dataset☆39Updated 3 years ago
- Learning to Initialize Neural Networks for Stable and Efficient Training☆135Updated 2 years ago
- Code for the CVPR 2021 paper: Understanding Failures of Deep Networks via Robust Feature Extraction☆35Updated 2 years ago
- Training a model similar to OpenAI DALL-E with volunteers from all over the Internet using hivemind and dalle-pytorch (NeurIPS 2021 demo)☆27Updated last year
- "Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices", official implementation☆28Updated last year
- Another attempt at a long-context / efficient transformer by me☆37Updated 2 years ago
- Collaborative inference of latent diffusion via hivemind☆13Updated last year
- Pruning applied to Facial Recognition.☆15Updated 5 years ago
- Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)☆116Updated 2 years ago
- Code Release for "Broken Neural Scaling Laws" (BNSL) paper☆57Updated last year
- PyTorch implementation of HashedNets☆36Updated last year
- Official repository for the paper "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers" (NeurIPS 2021)☆47Updated last year
- ☆35Updated 5 years ago
- Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).☆56Updated 2 years ago
- Official code for paper "Non-Adversarial Image Synthesis with Generative Latent Nearest Neighbors"☆28Updated 4 years ago
- Code to implement the AND-mask and geometric mean to do gradient based optimization, from the paper "Learning explanations that are hard …☆39Updated 3 years ago
- Code for ICLR 2021 Paper, "Anytime Sampling for Autoregressive Models via Ordered Autoencoding"☆24Updated last year
- Automatically take good care of your preemptible TPUs☆31Updated last year
- [AAAI 2020 Oral] Low-variance Black-box Gradient Estimates for the Plackett-Luce Distribution☆36Updated 3 years ago