facebookresearch / Mixture-of-TransformersLinks
Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models. TMLR 2025.
☆81Updated 2 months ago
Alternatives and similar repositories for Mixture-of-Transformers
Users that are interested in Mixture-of-Transformers are comparing it to the libraries listed below
Sorting:
- Esoteric Language Models☆87Updated 2 weeks ago
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun☆54Updated 4 months ago
- Official PyTorch Implementation for Vision-Language Models Create Cross-Modal Task Representations, ICML 2025☆27Updated 2 months ago
- ☆82Updated 10 months ago
- LL3M: Large Language and Multi-Modal Model in Jax☆72Updated last year
- Resa: Transparent Reasoning Models via SAEs☆39Updated last month
- $100K or 100 Days: Trade-offs when Pre-Training with Academic Resources☆140Updated last month
- One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation☆40Updated 9 months ago
- ☆33Updated 6 months ago
- Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch☆177Updated 3 weeks ago
- Official implementation of MAIA, A Multimodal Automated Interpretability Agent☆82Updated 3 weeks ago
- [ICCV 2025] Auto Interpretation Pipeline and many other functionalities for Multimodal SAE Analysis.☆144Updated this week
- Repository for the Q-Filters method (https://arxiv.org/pdf/2503.02812)☆33Updated 4 months ago
- [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"☆115Updated last week
- Code, results and other artifacts from the paper introducing the WildChat-50m dataset and the Re-Wild model family.☆29Updated 3 months ago
- The official implementation of Regularized Policy Gradient (RPG) (https://arxiv.org/abs/2505.17508)☆35Updated last week
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆60Updated 10 months ago
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆33Updated 3 months ago
- [Under Review] Official PyTorch implementation code for realizing the technical part of Phantom of Latent representing equipped with enla…☆60Updated 9 months ago
- Official implementation of ECCV24 paper: POA☆24Updated 11 months ago
- Remasking Discrete Diffusion Models with Inference-Time Scaling☆34Updated 4 months ago
- Easily run PyTorch on multiple GPUs & machines☆46Updated 2 weeks ago
- Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".☆128Updated 2 weeks ago
- 🦾 EvalGIM (pronounced as "EvalGym") is an evaluation library for generative image models. It enables easy-to-use, reproducible automatic…☆82Updated 6 months ago
- Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"☆29Updated 8 months ago
- [ICLR 2025] Source code for paper "A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Finegr…☆76Updated 7 months ago
- ☆17Updated 7 months ago
- Implementations of attention with the softpick function, naive and FlashAttention-2☆80Updated 2 months ago
- X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains☆46Updated 2 months ago
- Official implementation of "PyVision: Agentic Vision with Dynamic Tooling."☆24Updated this week