pixeli99 / MixLN
[ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxiang Li, Lu Yin, Shiwei Liu
☆21Updated 4 months ago
Alternatives and similar repositories for MixLN:
Users that are interested in MixLN are comparing it to the libraries listed below
- The official repo of continuous speculative decoding☆26Updated last month
- ☆41Updated 6 months ago
- ☆17Updated 4 months ago
- Code for "R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts"☆15Updated 2 months ago
- Triton implement of bi-directional (non-causal) linear attention☆47Updated 3 months ago
- [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models☆30Updated 10 months ago
- The open-source materials for paper "Sparsing Law: Towards Large Language Models with Greater Activation Sparsity".☆21Updated 5 months ago
- [NAACL 2025 Oral] Multimodal Needle in a Haystack (MMNeedle): Benchmarking Long-Context Capability of Multimodal Large Language Models☆42Updated last week
- A big_vision inspired repo that implements a generic Auto-Encoder class capable in representation learning and generative modeling.☆35Updated 10 months ago
- Official implementation of the paper "MMInA: Benchmarking Multihop Multimodal Internet Agents"☆43Updated 2 months ago
- This is the implementation of CounterCurate, the data curation pipeline of both physical and semantic counterfactual image-caption pairs.☆18Updated 10 months ago
- Code for "Merging Text Transformers from Different Initializations"☆20Updated 3 months ago
- Official implementation of ECCV24 paper: POA☆24Updated 9 months ago
- ☆17Updated 4 months ago
- PyTorch implementation of StableMask (ICML'24)☆12Updated 10 months ago
- Official implementation of "The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs"☆26Updated 2 weeks ago
- The code for "VISTA: Enhancing Long-Duration and High-Resolution Video Understanding by VIdeo SpatioTemporal Augmentation" [CVPR2025]☆15Updated 2 months ago
- Code for "C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing"☆14Updated last month
- ☆39Updated last month
- Code for NOLA, an implementation of "nola: Compressing LoRA using Linear Combination of Random Basis"☆54Updated 8 months ago
- [Under Review] Official PyTorch implementation code for realizing the technical part of Phantom of Latent representing equipped with enla…☆57Updated 7 months ago
- Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"☆26Updated 6 months ago
- ☆11Updated this week
- Official implementation of the ICML 2024 paper RoSA (Robust Adaptation)☆41Updated last year
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆29Updated last month
- ☆24Updated 3 weeks ago
- Official PyTorch Implementation for Vision-Language Models Create Cross-Modal Task Representations, ICML 2025☆22Updated last week
- ☆78Updated 8 months ago
- Exploration of automated dataset selection approaches at large scales.☆40Updated 2 months ago
- This repo is based on https://github.com/jiaweizzhao/GaLore☆27Updated 7 months ago