koayon / awesome-adaptive-computationLinks
A curated reading list of research in Adaptive Computation, Inference-Time Computation & Mixture of Experts (MoE).
☆152Updated 7 months ago
Alternatives and similar repositories for awesome-adaptive-computation
Users that are interested in awesome-adaptive-computation are comparing it to the libraries listed below
Sorting:
- Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch☆179Updated 2 months ago
- [NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models☆227Updated 3 months ago
- Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind☆177Updated 11 months ago
- ☆127Updated last year
- ☆202Updated 8 months ago
- This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.☆104Updated last year
- Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (NeurIPS 2024)☆192Updated last year
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆153Updated 2 months ago
- ☆102Updated 11 months ago
- ☆46Updated last year
- The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".☆174Updated 5 months ago
- ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward exp…☆223Updated last year
- Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"☆240Updated 2 months ago
- Understand and test language model architectures on synthetic tasks.☆222Updated last month
- ☆72Updated last year
- Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)☆161Updated 4 months ago
- ☆100Updated last year
- RuLES: a benchmark for evaluating rule-following in language models☆230Updated 6 months ago
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆170Updated last year
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆127Updated last year
- Language models scale reliably with over-training and on downstream tasks☆98Updated last year
- ☆194Updated 3 weeks ago
- ☆187Updated 4 months ago
- ☆141Updated 2 weeks ago
- [NeurIPS 2024] Low rank memory efficient optimizer without SVD☆30Updated 2 months ago
- ☆69Updated last year
- The official repo for "LLoCo: Learning Long Contexts Offline"☆118Updated last year
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆98Updated 11 months ago
- Some preliminary explorations of Mamba's context scaling.☆217Updated last year
- Code for NeurIPS'24 paper 'Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization'☆229Updated last month