AdityaNG / kan-gptLinks
The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling
☆720Updated 7 months ago
Alternatives and similar repositories for kan-gpt
Users that are interested in kan-gpt are comparing it to the libraries listed below
Sorting:
- Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines.☆380Updated last year
- ☆739Updated last year
- [ICLR 2025] Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling☆888Updated 2 months ago
- Variations of Kolmogorov-Arnold Networks☆115Updated last year
- Build high-performance AI models with modular building blocks☆533Updated this week
- This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing th…☆882Updated 3 months ago
- Training small GPT-2 style models using Kolmogorov-Arnold networks.☆120Updated last year
- Official repository for the paper "Grokfast: Accelerated Grokking by Amplifying Slow Gradients"☆555Updated last year
- ☆447Updated last year
- PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention…☆290Updated last year
- PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"☆172Updated 3 months ago
- Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"☆555Updated 6 months ago
- A comprehensive collection of KAN(Kolmogorov-Arnold Network)-related resources, including libraries, projects, tutorials, papers, and mor…☆2,995Updated 2 weeks ago
- Mamba-Chat: A chat LLM based on the state-space model architecture 🐍☆926Updated last year
- A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!☆1,123Updated 5 months ago
- An implementation of "Retentive Network: A Successor to Transformer for Large Language Models"☆1,193Updated last year
- An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).☆4,418Updated 11 months ago
- Open weights language model from Google DeepMind, based on Griffin.☆644Updated last month
- The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”☆962Updated last year
- A simple and efficient Mamba implementation in pure PyTorch and MLX.☆1,279Updated 7 months ago
- Annotated version of the Mamba paper☆486Updated last year
- Simple, minimal implementation of the Mamba SSM in one pytorch file. Using logcumsumexp (Heisen sequence).☆120Updated 8 months ago
- llama3.np is a pure NumPy implementation for Llama 3 model.☆986Updated 2 months ago
- First-principle implementations of groundbreaking AI algorithms using a wide range of deep learning frameworks, accompanied by supporting…☆174Updated last week
- Official repository of Evolutionary Optimization of Model Merging Recipes☆1,346Updated 7 months ago
- Beyond Language Models: Byte Models are Digital World Simulators☆324Updated last year
- PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily wri…☆1,375Updated this week
- The Multilayer Perceptron Language Model☆553Updated 11 months ago
- Reaching LLaMA2 Performance with 0.1M Dollars☆984Updated 11 months ago
- Schedule-Free Optimization in PyTorch☆2,189Updated last month