koayon / awesome-adaptive-computationLinks
A curated reading list of research in Adaptive Computation, Inference-Time Computation & Mixture of Experts (MoE).
☆154Updated 8 months ago
Alternatives and similar repositories for awesome-adaptive-computation
Users that are interested in awesome-adaptive-computation are comparing it to the libraries listed below
Sorting:
- Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (NeurIPS 2024)☆192Updated last year
- ☆202Updated 9 months ago
- ☆69Updated last year
- ☆128Updated last year
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆162Updated 2 months ago
- [NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models☆229Updated 4 months ago
- Understand and test language model architectures on synthetic tasks.☆225Updated 2 months ago
- Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch☆179Updated 3 months ago
- ☆104Updated 11 months ago
- Code for studying the super weight in LLM☆117Updated 9 months ago
- This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.☆104Updated last year
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆171Updated last year
- Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"☆241Updated 3 months ago
- Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)☆161Updated 5 months ago
- Some preliminary explorations of Mamba's context scaling.☆217Updated last year
- Implementation of CALM from the paper "LLM Augmented LLMs: Expanding Capabilities through Composition", out of Google Deepmind☆178Updated last year
- ☆196Updated 2 weeks ago
- ☆100Updated last year
- ☆46Updated last year
- Code accompanying the paper "Massive Activations in Large Language Models"☆179Updated last year
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆128Updated last year
- ☆85Updated last year
- ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward exp…☆225Updated this week
- ☆72Updated last year
- [NeurIPS 2024] Low rank memory efficient optimizer without SVD☆31Updated 2 months ago
- The official implementation of the paper "What Matters in Transformers? Not All Attention is Needed".☆176Updated 5 months ago
- ☆190Updated 5 months ago
- A MAD laboratory to improve AI architecture designs 🧪☆129Updated 9 months ago
- nanoGPT-like codebase for LLM training☆107Updated 4 months ago
- LLM-Merging: Building LLMs Efficiently through Merging☆203Updated 11 months ago