AdityaNG / kan-gpt
The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling
☆714Updated 4 months ago
Alternatives and similar repositories for kan-gpt:
Users that are interested in kan-gpt are comparing it to the libraries listed below
- Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines.☆368Updated 10 months ago
- ☆725Updated 10 months ago
- This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing th…☆850Updated 4 months ago
- An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).☆4,265Updated 7 months ago
- FastKAN: Very Fast Implementation of Kolmogorov-Arnold Networks (KAN)☆393Updated 9 months ago
- Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"☆855Updated last month
- Variations of Kolmogorov-Arnold Networks☆113Updated 10 months ago
- A comprehensive collection of KAN(Kolmogorov-Arnold Network)-related resources, including libraries, projects, tutorials, papers, and mor…☆2,826Updated last month
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆115Updated 9 months ago
- Schedule-Free Optimization in PyTorch☆2,116Updated 3 weeks ago
- An easy to use PyTorch implementation of the Kolmogorov Arnold Network and a few novel variations☆178Updated 4 months ago
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆160Updated last month
- Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793☆399Updated 3 months ago
- UNet diffusion model in pure CUDA☆600Updated 8 months ago
- First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…☆147Updated 2 weeks ago
- KAN for Vision Transformer☆245Updated 5 months ago
- Open weights language model from Google DeepMind, based on Griffin.☆628Updated last month
- Code release for DynamicTanh (DyT)☆710Updated this week
- Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"☆548Updated 2 months ago
- Beyond Language Models: Byte Models are Digital World Simulators☆322Updated 9 months ago
- Interactively inspect module inputs, outputs, parameters, and gradients.☆323Updated 2 months ago
- Official Implementation of "ADOPT: Modified Adam Can Converge with Any β2 with the Optimal Rate"☆419Updated 3 months ago
- Stanford NLP Python library for Representation Finetuning (ReFT)☆1,445Updated last month
- This project is dedicated to the implementation and research of Kolmogorov-Arnold convolutional networks. The repository includes implem…☆477Updated 4 months ago
- A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest…☆448Updated last week
- The official implementation of Self-Play Fine-Tuning (SPIN)☆1,135Updated 10 months ago
- From scratch implementation of a vision language model in pure PyTorch☆205Updated 10 months ago
- PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention…☆287Updated 10 months ago
- Official JAX implementation of Learning to (Learn at Test Time): RNNs with Expressive Hidden States☆398Updated 7 months ago
- Muon optimizer: +>30% sample efficiency with <3% wallclock overhead☆521Updated 2 weeks ago