ScaledFoundations / MatMamba
Code and pretrained models for the paper: "MatMamba: A Matryoshka State Space Model"
☆50Updated last month
Related projects ⓘ
Alternatives and complementary repositories for MatMamba
- DPO, but faster 🚀☆21Updated 2 weeks ago
- Official implementation of ECCV24 paper: POA☆24Updated 3 months ago
- ☆21Updated last week
- Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"☆49Updated 2 months ago
- Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mode…☆77Updated last month
- One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptation☆29Updated 3 weeks ago
- Implementation of Bitune: Bidirectional Instruction-Tuning☆15Updated 5 months ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆91Updated last month
- HGRN2: Gated Linear RNNs with State Expansion☆49Updated 2 months ago
- Official implementation of "Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Models"☆35Updated 10 months ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆46Updated 2 months ago
- Code and benchmark for the paper: "A Practitioner's Guide to Continual Multimodal Pretraining" [NeurIPS'24]☆33Updated 2 months ago
- Pytorch Implementation of the paper: "Learning to (Learn at Test Time): RNNs with Expressive Hidden States"☆23Updated last week
- Code repository for the public reproduction of the language modelling experiments on "MatFormer: Nested Transformer for Elastic Inference…☆18Updated 11 months ago
- Research on Tabular Foundation Models☆27Updated 2 months ago
- Implementation of the paper: "BRAVE : Broadening the visual encoding of vision-language models"☆21Updated this week
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆33Updated 8 months ago
- Official code repository for paper: "ExPLoRA: Parameter-Efficient Extended Pre-training to Adapt Vision Transformers under Domain Shifts"☆24Updated last month
- Official Implementation Of The Paper: `DeciMamba: Exploring the Length Extrapolation Potential of Mamba'☆20Updated 3 months ago
- [Under Review] Official PyTorch implementation code for realizing the technical part of Phantom of Latent representing equipped with enla…☆45Updated last month
- Collection of autoregressive model implementation☆66Updated last week
- ☆61Updated 2 months ago
- ☆29Updated this week
- ☆25Updated 2 months ago
- Learning to Retrieve by Trying - Source code for Grounding by Trying: LLMs with Reinforcement Learning-Enhanced Retrieval☆24Updated last week
- ☆38Updated 3 months ago
- ☆40Updated this week
- Official Code Repository for EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents (COLM 2024)☆25Updated 3 months ago
- ☆34Updated 8 months ago
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆36Updated 11 months ago