miniHuiHui / awesome-high-order-neural-networkLinks
☆49Updated 8 months ago
Alternatives and similar repositories for awesome-high-order-neural-network
Users that are interested in awesome-high-order-neural-network are comparing it to the libraries listed below
Sorting:
- A library for calculating the FLOPs in the forward() process based on torch.fx☆112Updated 2 months ago
- ☆191Updated last year
- Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficien…☆104Updated 2 months ago
- Collection of papers on state-space models☆593Updated last month
- [ICML 2024] Official code for the paper "Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark ".☆103Updated 11 months ago
- This is a list of peer-reviewed representative papers on deep learning dynamics (optimization dynamics of neural networks). The success o…☆276Updated last year
- tinybig for deep function learning☆60Updated 5 months ago
- The official GitHub page for the survey paper "A Survey on Mixture of Experts in Large Language Models".☆361Updated 2 months ago
- PyTorch implementation of the Differential-Transformer architecture for sequence modeling, specifically tailored as a decoder-only model …☆66Updated 7 months ago
- Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"☆55Updated 2 months ago
- A lecture note for understanding deep learning☆319Updated last month
- Minimal Mamba-2 implementation in PyTorch☆198Updated 11 months ago
- A repository for DenseSSMs☆87Updated last year
- Inference Speed Benchmark for Learning to (Learn at Test Time): RNNs with Expressive Hidden States☆67Updated 10 months ago
- Offical implementation of "MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map" (NeurIPS2024 Oral)☆24Updated 4 months ago
- This repository periodicly updates the MTL paper and resources☆55Updated last month
- EasyLiterature is an open-sourced, Python-based command line tool for automatic literature management.☆278Updated 9 months ago
- ☆145Updated 8 months ago
- [ICML 2022, Oral] The PyTorch Implementation of Adaptive Inertia Methods. The algorithms are based on our paper: "Adaptive Inertia: Dise…☆150Updated 2 years ago
- ☆60Updated 4 months ago
- ☆248Updated 9 months ago
- A More Fair and Comprehensive Comparison between KAN and MLP☆169Updated 9 months ago
- ☆198Updated 7 months ago
- Efficient 2:4 sparse training algorithms and implementations☆54Updated 5 months ago
- [ICML 2025] Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization☆69Updated 4 months ago
- [ICLR 2025] Official PyTorch Implementation of Gated Delta Networks: Improving Mamba2 with Delta Rule☆167Updated 2 months ago
- The official implementation of Tensor ProducT ATTenTion Transformer (T6) (https://arxiv.org/abs/2501.06425)☆373Updated 2 weeks ago
- A generalized framework for subspace tuning methods in parameter efficient fine-tuning.☆141Updated 3 months ago
- Official PyTorch Implementation for Fast Adaptive Multitask Optimization (FAMO)☆86Updated last year
- An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivatio…☆89Updated last year