a-r-r-o-w / kanformerLinks
Naively combining transformers and Kolmogorov-Arnold Networks to learn and experiment
☆34Updated 10 months ago
Alternatives and similar repositories for kanformer
Users that are interested in kanformer are comparing it to the libraries listed below
Sorting:
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆169Updated 2 months ago
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆54Updated last year
- ☆133Updated last year
- Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling☆194Updated 2 months ago
- my attempts at implementing various bits of Sepp Hochreiter's new xLSTM architecture☆129Updated last year
- Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…☆105Updated 2 months ago
- Code repository for Black Mamba☆246Updated last year
- Implementation of Agent Attention in Pytorch☆90Updated 10 months ago
- Trying out the Mamba architecture on small examples (cifar-10, shakespeare char level etc.)☆46Updated last year
- First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…☆167Updated this week
- Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"☆118Updated 2 months ago
- Variations of Kolmogorov-Arnold Networks☆114Updated last year
- A modified CNN architecture using Kolmogorov-Arnold Networks☆80Updated last year
- Official PyTorch Implementation of "The Hidden Attention of Mamba Models"☆222Updated last year
- KAN for Vision Transformer☆247Updated 7 months ago
- An easy to use PyTorch implementation of the Kolmogorov Arnold Network and a few novel variations☆181Updated 6 months ago
- Awesome list of papers that extend Mamba to various applications.☆133Updated 2 months ago
- Kolmogorov–Arnold Networks with modified activation (using MLP to represent the activation)☆105Updated 7 months ago
- A More Fair and Comprehensive Comparison between KAN and MLP☆169Updated 9 months ago
- Benchmarking and Testing FastKAN☆77Updated last year
- Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"☆55Updated 2 months ago
- A repository for DenseSSMs☆87Updated last year
- Benchmark for efficiency in memory and time of different KAN implementations.☆124Updated 9 months ago
- Collection of tests performed during the study of the new Kolmogorov-Arnold Neural Networks (KAN)☆39Updated 3 months ago
- Explorations into the proposal from the paper "Grokfast, Accelerated Grokking by Amplifying Slow Gradients"☆100Updated 5 months ago
- ☆43Updated 4 months ago
- Cuda implementation of Extended Long Short Term Memory (xLSTM) with C++ and PyTorch ports☆87Updated 11 months ago
- Implementation of Infini-Transformer in Pytorch☆111Updated 5 months ago
- Pytorch implementation of the xLSTM model by Beck et al. (2024)☆165Updated 9 months ago
- working implimention of deepseek MLA☆41Updated 4 months ago