Mixture of A Million Experts
☆54Jul 30, 2024Updated last year
Alternatives and similar repositories for PEER
Users that are interested in PEER are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆135Nov 1, 2025Updated 4 months ago
- ☆12Dec 22, 2024Updated last year
- BFloat16 Fused Adam Operator for PyTorch☆17Nov 16, 2024Updated last year
- Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆116Updated this week
- sigma-MoE layer☆21Jan 5, 2024Updated 2 years ago
- Bare Metal GPUs on DigitalOcean Gradient AI • AdPurpose-built for serious AI teams training foundational models, running large-scale inference, and pushing the boundaries of what's possible.
- code for the paper, "flexible multitask computation in recurrent networks utilizes shared dynamical motifs"☆10Mar 2, 2025Updated last year
- Experiments on the impact of depth in transformers and SSMs.☆41Oct 23, 2025Updated 5 months ago
- Xmixers: A collection of SOTA efficient token/channel mixers☆28Sep 4, 2025Updated 6 months ago
- minGPT in JAX☆48Jan 10, 2022Updated 4 years ago
- The official implementation of HybridNorm: Towards Stable and Efficient Transformer Training via Hybrid Normalization☆18Mar 7, 2025Updated last year
- [EMNLP 2025] Circuit-Aware Editing Enables Generalizable Knowledge Learners☆19Nov 17, 2025Updated 4 months ago
- ☆15Jul 13, 2025Updated 8 months ago
- ☆17Jun 11, 2025Updated 9 months ago
- Official Pytorch implementation of Chromatic Graph Transformers☆10Jun 14, 2023Updated 2 years ago
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Clustered Compositional Embeddings☆11Oct 25, 2023Updated 2 years ago
- ☆18Jun 23, 2025Updated 9 months ago
- Code and dataset for NAACL 2022 paper "CoSIm: Commonsense Reasoning for Counterfactual Scene Imagination" Hyounghun Kim, Abhay Zala, Mohi…☆16Nov 26, 2022Updated 3 years ago
- codes and plots for "Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs"☆10Dec 30, 2024Updated last year
- ☆36Jan 25, 2026Updated 2 months ago
- Multi-Layer Sparse Autoencoders (ICLR 2025)☆29Feb 6, 2026Updated last month
- Implementation of Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems☆14Nov 11, 2023Updated 2 years ago
- A Zen approach to configuring your Python project☆16Feb 27, 2026Updated last month
- ☆12Sep 16, 2024Updated last year
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆101Sep 30, 2024Updated last year
- "Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding" Zhenyu Zhang, Runjin Chen, Shiw…☆31May 7, 2024Updated last year
- ☆13May 9, 2024Updated last year
- ☆28Oct 2, 2025Updated 5 months ago
- Unofficial implementation for the paper "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆175Jun 20, 2024Updated last year
- This github repository hosts the code used within my thesis work and my last publication.☆12Jul 20, 2017Updated 8 years ago
- Audio Demo for "FastSVC: Fast Cross-Domain Singing Voice Conversion with Feature-wise Linear Modulation"☆21Apr 7, 2021Updated 4 years ago
- ☆10Jun 27, 2024Updated last year
- ☆16Jul 8, 2024Updated last year
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Source code for ICDE 2020 paper Collective Entity Alignment via Adaptive Features (CEA).☆15Jun 10, 2020Updated 5 years ago
- ☆12Jan 17, 2024Updated 2 years ago
- Find context neurons in Pythia models.☆13Jun 13, 2023Updated 2 years ago
- Code for experiments on transformers using Markovian data.☆22Nov 22, 2024Updated last year
- Official pytorch code for "APP: Anytime Progressive Pruning" (DyNN @ ICML, 2022; CLL @ ACML, 2022, SNN @ ICML, 2022 and SlowDNN 2023)☆16Nov 22, 2022Updated 3 years ago
- TIPS (ICLR'25): Text-Image Pretraining with Spatial Awareness☆118Apr 9, 2025Updated 11 months ago
- Personal solutions to the Triton Puzzles☆20Jul 18, 2024Updated last year