HazyResearch / mongoose
A Learnable LSH Framework for Efficient NN Training
☆31Updated 3 years ago
Alternatives and similar repositories for mongoose:
Users that are interested in mongoose are comparing it to the libraries listed below
- "Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices", official implementation☆29Updated last month
- Fast training of unitary deep network layers from low-rank updates☆29Updated 2 years ago
- ☆15Updated 2 years ago
- Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing☆48Updated 3 years ago
- Successfully training approximations to full-rank matrices for efficiency in deep learning.☆17Updated 4 years ago
- Make triton easier☆47Updated 9 months ago
- A custom PyTorch layer that is capable of implementing extremely wide and sparse linear layers efficiently☆49Updated last year
- Distributed ML Optimizer☆30Updated 3 years ago
- Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent☆14Updated 2 years ago
- [NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling☆36Updated last year
- ☆27Updated 2 months ago
- Official repository for the paper "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers" (NeurIPS 2021)☆48Updated 2 years ago
- Spartan is an algorithm for training sparse neural network models. This repository accompanies the paper "Spartan Differentiable Sparsity…☆24Updated 2 years ago
- Sparsity support for PyTorch☆35Updated last week
- ☆26Updated last year
- Distributed K-FAC Preconditioner for PyTorch☆85Updated last week
- Codes of the paper Deformable Butterfly: A Highly Structured and Sparse Linear Transform.☆12Updated 3 years ago
- Code accompanying our paper "Feature Learning in Infinite-Width Neural Networks" (https://arxiv.org/abs/2011.14522)☆61Updated 3 years ago
- Meta-learning inductive biases in the form of useful conserved quantities.☆37Updated 2 years ago
- ☆13Updated this week
- Source-to-Source Debuggable Derivatives in Pure Python☆15Updated last year
- Some microbenchmarks and design docs before commencement☆12Updated 4 years ago
- Implementation for ACProp ( Momentum centering and asynchronous update for adaptive gradient methdos, NeurIPS 2021)☆15Updated 3 years ago
- ☆29Updated 2 years ago
- RWKV model implementation☆37Updated last year
- ☆38Updated last year
- Using FlexAttention to compute attention with different masking patterns☆42Updated 6 months ago
- JAX implementation of "Fine-Tuning Language Models with Just Forward Passes"☆19Updated last year
- 👑 Pytorch code for the Nero optimiser.☆20Updated 2 years ago
- ☆11Updated 2 years ago