Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
☆115Mar 3, 2026Updated last week
Alternatives and similar repositories for Mixture-of-Depths
Users that are interested in Mixture-of-Depths are comparing it to the libraries listed below
Sorting:
- An unofficial implementation of "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆36Jun 7, 2024Updated last year
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆177Jun 20, 2024Updated last year
- Google DeepMind: Mixture of Depths Unofficial Implementation.☆12May 29, 2024Updated last year
- A swarm of LLM agents that will help you test, document, and productionize your code!☆16Feb 16, 2026Updated 3 weeks ago
- Implementation of SoundtStream from the paper: "SoundStream: An End-to-End Neural Audio Codec"☆13Jan 27, 2025Updated last year
- Implementation of a Hierarchical Mamba as described in the paper: "Hierarchical State Space Models for Continuous Sequence-to-Sequence Mo…☆15Nov 11, 2024Updated last year
- Implementation of the Pairformer model used in AlphaFold 3☆14Mar 2, 2026Updated last week
- A sophisticated multi-agent system designed for real-time market analysis of HTX (formerly Huobi) exchange data. This swarm combines spec…☆10Mar 18, 2025Updated 11 months ago
- Various agents from all of the top agent frameworks to integrate into swarms! Langchain, Griptape, CrewAI, and more!☆18Dec 22, 2025Updated 2 months ago
- Brainwave is a state-of-the-art neural decoder that transforms electroencephalogram (EEG) and brain signals into multimodal outputs inclu…☆14Oct 6, 2025Updated 5 months ago
- OmniByteFormer is a generalized Transformer model that can process any type of data by converting it into byte sequences, bypassing tradi…☆15Mar 2, 2026Updated last week
- AgentParse is a high-performance parsing library designed to map various structured data formats (such as Pydantic models, JSON, YAML, an…☆18Oct 13, 2025Updated 4 months ago
- An implementation of the paper Brain2Qwerty that translates brain EEG data into text for reading people's brains. There was no code so we…☆22Feb 9, 2025Updated last year
- An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO☆29Mar 3, 2026Updated last week
- AgentOS is a lightweight, single-file implementation that provides a robust foundation for building autonomous AI agents. It implements t…☆22Jul 11, 2025Updated 7 months ago
- A forest of autonomous agents.☆19Jan 27, 2025Updated last year
- NewsAgent is an enterprise-grade news aggregation agent designed to fetch, query, and summarize news from multiple sources at scale.☆27Oct 13, 2025Updated 4 months ago
- Mamba R1 represents a novel architecture that combines the efficiency of Mamba's state space models with the scalability of Mixture of Ex…☆25Oct 13, 2025Updated 4 months ago
- Graph Transformers for Large Graphs☆22Apr 26, 2024Updated last year
- Repository for initial POC NLP based SQL adapter using LLM.☆10May 6, 2025Updated 10 months ago
- Transform unstructured documents into actionable, structured data with enterprise-grade precision and reliability, ready for large-scale …☆20Oct 13, 2025Updated 4 months ago
- Community Implementation of the paper: "Multi-Head Mixture-of-Experts" In PyTorch☆29Jan 31, 2026Updated last month
- bulk image downloader freeware, reddit bulk image downloader, bulk image downloader extension, bulk image downloader from url, bulk image…☆25Feb 19, 2026Updated 2 weeks ago
- Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…☆120Jan 31, 2026Updated last month
- A framework making it effortless to convert any llm model into a reasoning agent like o1 or DeepSeek's r1☆24Oct 13, 2025Updated 4 months ago
- ☆63Oct 17, 2023Updated 2 years ago
- Implementation of a modular, high-performance, and simplistic mamba for high-speed applications☆40Nov 11, 2024Updated last year
- The Swarm Ecosystem☆27Aug 1, 2024Updated last year
- Paper dataset for "Factored Verification: Detecting and Reducing Hallucination in Summaries of Academic Papers"☆12Oct 20, 2024Updated last year
- ☆81Mar 11, 2025Updated 11 months ago
- A simple package for leveraging Falcon 180B and the HF ecosystem's tools, including training/inference scripts, safetensors, integrations…☆12Mar 11, 2024Updated last year
- A benchmark of programming tasks for LLMs that supports almost any programming language.☆13Jun 30, 2025Updated 8 months ago
- [NeurIPS 2025@FoRLM] R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search☆17Jan 24, 2026Updated last month
- Implementation of SelfExtend from the paper "LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning" from Pytorch and Zeta☆13Nov 11, 2024Updated last year
- Mixture of Expert (MoE) techniques for enhancing LLM performance through expert-driven prompt mapping and adapter combinations.☆12Feb 11, 2024Updated 2 years ago
- This repository serves as a central hub for discovering tools and services focused on automated prompt engineering. Whether you're lookin…☆14Oct 11, 2024Updated last year
- Zeta implementation of a reusable and plug in and play feedforward from the paper "Exponentially Faster Language Modeling"☆16Nov 11, 2024Updated last year
- ☆16Jul 29, 2025Updated 7 months ago
- Pytorch implementation of our paper accepted by ICML 2024 -- CaM: Cache Merging for Memory-efficient LLMs Inference☆48Jun 19, 2024Updated last year