a-r-r-o-w / kanformer
Naively combining transformers and Kolmogorov-Arnold Networks to learn and experiment
☆33Updated 3 months ago
Related projects ⓘ
Alternatives and complementary repositories for kanformer
- A modified CNN architecture using Kolmogorov-Arnold Networks☆65Updated 6 months ago
- An easy to use PyTorch implementation of the Kolmogorov Arnold Network and a few novel variations☆160Updated 3 months ago
- Benchmark for efficiency in memory and time of different KAN implementations.☆111Updated 2 months ago
- Benchmarking and Testing FastKAN☆65Updated 5 months ago
- Variations of Kolmogorov-Arnold Networks☆111Updated 6 months ago
- Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling☆170Updated last week
- Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines.☆349Updated 6 months ago
- This repository contains a better implementation of Kolmogorov-Arnold networks☆59Updated 6 months ago
- First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…☆70Updated last week
- A More Fair and Comprehensive Comparison between KAN and MLP☆150Updated 3 months ago
- ☆119Updated 6 months ago
- ☆77Updated 5 months ago
- Trying out the Mamba architecture on small examples (cifar-10, shakespeare char level etc.)☆42Updated 11 months ago
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆108Updated 5 months ago
- my attempts at implementing various bits of Sepp Hochreiter's new xLSTM architecture☆130Updated 6 months ago
- Implementation of Agent Attention in Pytorch☆86Updated 4 months ago
- Kolmogorov-Arnold Networks with various basis functions like B-Splines, Fourier, Chebyshev, Wavelets etc☆31Updated 6 months ago
- Kolmogorov–Arnold Networks with modified activation (using MLP to represent the activation)☆103Updated 3 weeks ago
- FastKAN: Very Fast Implementation of Kolmogorov-Arnold Networks (KAN)☆367Updated 5 months ago
- Pytorch (Lightning) implementation of the Mamba model☆14Updated 7 months ago
- Pytorch implementation of the xLSTM model by Beck et al. (2024)☆141Updated 3 months ago
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆137Updated last week
- Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"☆106Updated last week
- Kolmogorov-Arnold Networks (KAN) using Jacobi polynomials instead of B-splines.☆32Updated 6 months ago
- Understanding Kolmogorov-Arnold Networks: A Tutorial Series on KAN using Toy Examples☆166Updated last month
- My implementation of the original transformer model (Vaswani et al.). I've additionally included the playground.py file for visualizing o…☆41Updated 11 months ago
- Transformer model based on Kolmogorov–Arnold Network(KAN), which is an alternative of Multi-Layer Perceptron(MLP)☆25Updated last week
- A easy, reliable, fluid template for python packages complete with docs, testing suites, readme's, github workflows, linting and much muc…☆146Updated last week
- Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mode…☆79Updated 2 months ago
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆50Updated 7 months ago