An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
☆35Jun 7, 2024Updated 2 years ago
Alternatives and similar repositories for mixture-of-depths
Users that are interested in mixture-of-depths are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆176Jun 20, 2024Updated 2 years ago
- Scratchpad/Chain-of-Thought Prompts☆12Jun 6, 2022Updated 4 years ago
- My fork os allen AI's OLMo for educational purposes.☆29Dec 5, 2024Updated last year
- Unofficial implementations of block/layer-wise pruning methods for LLMs.☆78Apr 29, 2024Updated 2 years ago
- ☆24Jul 7, 2023Updated 3 years ago
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Simple Implementation of TinyGPTV in super simple Zeta lego blocks☆16Nov 11, 2024Updated last year
- Accelerating Multitask Training Trough Adaptive Transition [Efficient ML Model]☆12May 23, 2025Updated last year
- Official repository for paper "Can Language Models Follow Multiple Turns of Entangled Instructions?"☆13Sep 20, 2025Updated last year
- Code for ICLR 2025 publication☆11May 3, 2025Updated last year
- Quasar Attention Continuous-Time Memory Dynamics for Scaling Models Context to Millions of Tokens☆30Mar 26, 2026Updated 7 months ago
- TiledLower is a Dataflow Analysis and Codegen Framework written in Rust.☆13Nov 23, 2024Updated last year
- Repo hosting codes and materials related to speeding LLMs' inference using token merging.☆37Oct 9, 2025Updated last year
- ☆13May 22, 2023Updated 3 years ago
- Community Implementation of the paper: "Multi-Head Mixture-of-Experts" In PyTorch☆31Oct 26, 2026Updated last week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Repo for Rho-1: Token-level Data Selection & Selective Pretraining of LLMs.☆475Apr 18, 2024Updated 2 years ago
- Official PyTorch code release for Implicit Gradient Transport, NeurIPS'19☆21Jun 11, 2019Updated 7 years ago
- ☆14Aug 25, 2026Updated 2 months ago
- ☆13Mar 26, 2024Updated 2 years ago
- [CoRL 2026] Official Codebase for "Native Memory Compression for Long-Horizon Robotic Manipulation"☆18Oct 19, 2026Updated 2 weeks ago
- Collection of autoregressive model implementation☆85Oct 1, 2026Updated last month
- ☆14Aug 6, 2025Updated last year
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"☆94Oct 30, 2024Updated 2 years ago
- ☆69Mar 21, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Math-OCR-Zero: A high-quality mathematical dataset for reinforcement learning in multimodal large models☆12Mar 30, 2026Updated 7 months ago
- Official implementation of ICML'25 paper "Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddings"☆12Jun 8, 2025Updated last year
- Phone Agent: iOS AI assistant with ESP32-S3 BLE mouse, keyboard and GPIO48 RGB firmware☆21Oct 6, 2026Updated last month
- ☆15Jan 14, 2026Updated 9 months ago
- Video Summarization Transformer: Implementation in PyTorch of the Transformer model for video summarisation☆10Oct 27, 2020Updated 6 years ago
- ☆16Jul 29, 2025Updated last year
- Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)☆67Sep 28, 2024Updated 2 years ago
- Code accompanying the paper "Noise Contrastive Alignment of Language Models with Explicit Rewards" (NeurIPS 2024)☆59Nov 8, 2024Updated last year
- Training GPTs to solve interaction nets☆19Aug 14, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- codebase for Coruscant: Co-Designing GPU Kernel and Sparse Tensor Core to Advocate Unstructured Sparsity in Efficient LLM Inference☆11Oct 17, 2025Updated last year
- ☆13Nov 2, 2022Updated 4 years ago
- Implementation of "Learning Variable Ordering Heuristics for Solving Constraint Satisfaction Problems"☆11Feb 14, 2022Updated 4 years ago
- A transformer model that should be able to solve a simple NER task☆11Mar 7, 2019Updated 7 years ago
- 🔬 [Discrete Diffusion Survey] This is the curated paper list collected for our survey paper: "Discrete Diffusion Models: A Unified Frame…☆13Oct 13, 2026Updated 3 weeks ago
- Deep Dynamic Transformer Model for Multi-Start Team Orienteering Problem☆11Jul 15, 2022Updated 4 years ago
- [ICLR 2026] Generalizable Heuristic Generation Through LLMs with Meta-Optimization☆15Aug 23, 2026Updated 2 months ago