a-r-r-o-w / kanformer
Naively combining transformers and Kolmogorov-Arnold Networks to learn and experiment
☆31Updated last month
Related projects: ⓘ
- Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling☆153Updated last week
- my attempts at implementing various bits of Sepp Hochreiter's new xLSTM architecture☆121Updated 4 months ago
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆120Updated last week
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆105Updated 3 months ago
- KAN for Vision Transformer☆199Updated 3 months ago
- A modified CNN architecture using Kolmogorov-Arnold Networks☆59Updated 3 months ago
- Trying out the Mamba architecture on small examples (cifar-10, shakespeare char level etc.)☆40Updated 9 months ago
- This repository contains a better implementation of Kolmogorov-Arnold networks☆59Updated 4 months ago
- ☆111Updated 4 months ago
- An easy to use PyTorch implementation of the Kolmogorov Arnold Network and a few novel variations☆147Updated last month
- Variations of Kolmogorov-Arnold Networks☆112Updated 4 months ago
- Pytorch implementation of the xLSTM model by Beck et al. (2024)☆116Updated last month
- ☆72Updated 3 months ago
- Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines.☆336Updated 4 months ago
- A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (http…☆101Updated 9 months ago
- FastKAN: Very Fast Implementation of Kolmogorov-Arnold Networks (KAN)☆332Updated 3 months ago
- A single repo with all scripts and utils to train / fine-tune the Mamba model with or without FIM☆46Updated 5 months ago
- Build high-performance AI models with modular building blocks☆377Updated this week
- Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"☆48Updated last week
- Benchmarking and Testing FastKAN☆57Updated 3 months ago
- Kolmogorov-Arnold Transformer: A PyTorch Implementation with CUDA kernel☆221Updated this week
- Benchmark for efficiency in memory and time of different KAN implementations.☆104Updated 3 weeks ago
- Code repository for Black Mamba☆218Updated 7 months ago
- Official PyTorch Implementation of "The Hidden Attention of Mamba Models"☆186Updated 3 months ago
- Kolmogorov-Arnold Networks with various basis functions like B-Splines, Fourier, Chebyshev, Wavelets etc☆30Updated 4 months ago
- Implementation of Agent Attention in Pytorch☆83Updated 2 months ago
- LORA: Low-Rank Adaptation of Large Language Models implemented using PyTorch☆72Updated last year
- Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"☆90Updated last week
- Official implementation of Phi-Mamba. A MOHAWK-distilled model (Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Mode…☆61Updated last week
- Kolmogorov–Arnold Networks with modified activation (using fully connected network to represent the activation)☆98Updated 4 months ago