BICLab / MetaLA
Offical implementation of "MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map" (NeurIPS2024 Oral)
☆18Updated last week
Alternatives and similar repositories for MetaLA:
Users that are interested in MetaLA are comparing it to the libraries listed below
- A repository for DenseSSMs☆86Updated 9 months ago
- [ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retenti…☆60Updated 9 months ago
- This project contains code for the paper titled "SpikingBERT: Distilling BERT to Train Spiking Language Models Using Implicit Differentia…☆15Updated 11 months ago
- A Triton Kernel for incorporating Bi-Directionality in Mamba2☆60Updated last month
- Triton implement of bi-directional (non-causal) linear attention☆38Updated 2 weeks ago
- Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficien…☆69Updated this week
- [ICCV 23]An approach to enhance the efficiency of Vision Transformer (ViT) by concurrently employing token pruning and token merging tech…☆92Updated last year
- [WACV 2024] Spiking Denoising Diffusion Probabilistic Models☆40Updated 9 months ago
- Offical code of "QKFormer: Hierarchical Spiking Transformer using Q-K Attention" (NeurIPS 2024，Spotlight 3%)☆91Updated 3 weeks ago
- [ICLR 2024] Hebbian Learning based Orthogonal Projection for Continual Learning of Spiking Neural Networks☆31Updated 11 months ago
- [ICCV-23] Masked Spiking Transformer☆30Updated last year
- [CVPR'23] SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer☆62Updated 9 months ago
- [ICML 2024] When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models☆28Updated 7 months ago
- An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivatio…☆78Updated 10 months ago
- [ICML 2024] Official PyTorch implementation of "SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-paramete…☆86Updated 5 months ago
- ☆47Updated last year
- ☆92Updated 6 months ago
- Official implementation of "Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers"☆119Updated 5 months ago
- Offical implementation of "Spike-driven Transformer" (NeurIPS2023)☆235Updated 10 months ago
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆59Updated last year
- This repository is the official implementation of our Autoregressive Pretraining with Mamba in Vision☆68Updated 7 months ago
- HGRN2: Gated Linear RNNs with State Expansion☆52Updated 5 months ago
- [ICML 2024] CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformers.☆30Updated last month
- Offical implementation of "Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation …☆159Updated 8 months ago
- A library for calculating the FLOPs in the forward() process based on torch.fx☆95Updated 4 months ago
- The official implementation for MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learning☆39Updated 6 months ago
- [Neurips 2022] “ Back Razor: Memory-Efficient Transfer Learning by Self-Sparsified Backpropogation”, Ziyu Jiang*, Xuxi Chen*, Xueqin Huan…☆19Updated last year
- [CVPR'24] Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities☆98Updated 10 months ago
- Official implementation of "SViT: Revisiting Token Pruning for Object Detection and Instance Segmentation"☆28Updated last year
- [ICLR 2024 Spotlight] This is the official PyTorch implementation of "EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Di…☆55Updated 7 months ago