Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
☆124Oct 26, 2026Updated last week
Alternatives and similar repositories for Mixture-of-Depths
Users that are interested in Mixture-of-Depths are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆35Jun 7, 2024Updated 2 years ago
- Zeta implemantion of "Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Trans…☆12Updated this week
- A swarm of LLM agents that will help you test, document, and productionize your code!☆20Oct 26, 2026Updated last week
- OmniByteFormer is a generalized Transformer model that can process any type of data by converting it into byte sequences, bypassing tradi…☆17Updated this week
- An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO☆32Oct 26, 2026Updated last week
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- Community Implementation of the paper: "Multi-Head Mixture-of-Experts" In PyTorch☆31Oct 26, 2026Updated last week
- A simple and minimal open source implementation of "Introducing LFM2: The Fastest On-Device Foundation Models on the Market" from Liquid …☆34Oct 26, 2026Updated last week
- Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch☆29Oct 26, 2026Updated last week
- Automate the creation of high quality research papers in latex. Powered by Swarms 🤖☆11Dec 1, 2025Updated 11 months ago
- Open-sourcing code associated with the AAAI-25 paper "On the Expressiveness and Length Generalization of Selective State-Space Models on …☆17Sep 28, 2026Updated last month
- An open source community implementation of the model MELLE from the paper: "Autoregressive Speech Synthesis without Vector Quantization"☆16Updated this week
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆137Nov 1, 2025Updated last year
- AgentParse is a high-performance parsing library designed to map various structured data formats (such as Pydantic models, JSON, YAML, an…☆18Oct 13, 2025Updated last year
- [ICML 2025] Official implementation of the paper "SkipGPT: Dynamic Layer Pruning Reinvented with Token Awareness and Module Decoupling". …☆21Nov 17, 2025Updated 11 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- [ICLR 2023] "Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers" by Tianlong Chen*, Zhenyu Zhang*, Ajay Jaiswal…☆56Feb 28, 2023Updated 3 years ago
- Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…☆130Oct 26, 2026Updated last week
- An open source implementation of R1☆31Oct 26, 2026Updated last week
- Official Implement for Paper "Neural Krylov Iteration for Accelerating Linear System Solving"☆16May 6, 2025Updated last year
- Solving Token Gradient Conflict in Mixture-of-Experts for Large Vision-Language Model☆13Feb 11, 2025Updated last year
- Welcome to Prometheus, the revolutionary AI model that allows you to generate DNA sequences for any creature you can imagine. Whether it’…☆17Updated this week
- Implementation of "Audio xLSTMs: Learning Self-supervised audio representations with xLSTMs" in PyTorch☆20Oct 26, 2026Updated last week
- Mixture of A Million Experts☆64Jul 30, 2024Updated 2 years ago
- OLMoE: Open Mixture-of-Experts Language Models☆1,102Sep 23, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- Generate High Quality textual or multi-modal datasets with Agents☆18Jun 7, 2023Updated 3 years ago
- Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference☆51Jun 19, 2024Updated 2 years ago
- ☆64Oct 17, 2023Updated 3 years ago
- [ICLR 2025] MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Experts☆281Oct 16, 2024Updated 2 years ago
- Simple Implementation of a Transformer in the new framework MLX by Apple☆19Nov 18, 2024Updated last year
- A sophisticated multi-agent system designed for real-time market analysis of HTX (formerly Huobi) exchange data. This swarm combines spec…☆11Mar 18, 2025Updated last year
- CogNetX is an advanced, multimodal neural network architecture inspired by human cognition. It integrates speech, vision, and video proce…☆22Oct 26, 2026Updated last week
- DPO, but faster 🚀☆51Dec 6, 2024Updated last year
- Implementation of MambaFormer in Pytorch ++ Zeta from the paper: "Can Mamba Learn How to Learn? A Comparative Study on In-Context Learnin…☆23Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models☆142Jun 12, 2024Updated 2 years ago
- Implementation of a Hierarchical Mamba as described in the paper: "Hierarchical State Space Models for Continuous Sequence-to-Sequence Mo…☆16Nov 11, 2024Updated last year
- This repository provides a minimal, single-file implementation of SingLoRA (Single Matrix Low-Rank Adaptation) as described in the paper …☆47Oct 26, 2026Updated last week
- Build high-performance AI models with modular building blocks☆605Oct 26, 2026Updated last week
- ☆15Jul 15, 2023Updated 3 years ago
- ☆16Oct 31, 2024Updated 2 years ago
- [ACL 2026] G2RPO-A: Guided Group Relative Policy Optimization with Adaptive Guidance☆16Sep 15, 2026Updated last month