Normalized Transformer (nGPT)
☆198Nov 19, 2024Updated last year
Alternatives and similar repositories for ngpt
Users that are interested in ngpt are comparing it to the libraries listed below
Sorting:
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆237Jun 15, 2025Updated 8 months ago
- an open source reproduction of NVIDIA's nGPT (Normalized Transformer with Representation Learning on the Hypersphere)☆110Mar 7, 2025Updated 11 months ago
- Implementation of Hyena Hierarchy in JAX☆10Apr 30, 2023Updated 2 years ago
- Code for the paper "Function-Space Learning Rates"☆25Jun 3, 2025Updated 8 months ago
- Code for the paper "Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns"☆18Mar 15, 2024Updated last year
- ☆67Mar 21, 2025Updated 11 months ago
- NanoGPT (124M) in 2 minutes☆4,679Updated this week
- Create synthetic datasets from scratch using AI-powered generation. Define topics, customize prompts, and generate high-quality reasoning…☆29Feb 14, 2026Updated 2 weeks ago
- TiledLower is a Dataflow Analysis and Codegen Framework written in Rust.☆14Nov 23, 2024Updated last year
- Mamba support for transformer lens☆19Sep 17, 2024Updated last year
- ☆24May 23, 2025Updated 9 months ago
- Code for the paper "Cottention: Linear Transformers With Cosine Attention"☆20Nov 15, 2025Updated 3 months ago
- Fork of Flame repo for training of some new stuff in development☆19Feb 20, 2026Updated last week
- Fluid Language Model Benchmarking☆26Sep 16, 2025Updated 5 months ago
- Parallel Associative Scan for Language Models☆18Jan 8, 2024Updated 2 years ago
- ☆45Apr 30, 2018Updated 7 years ago
- [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruning☆140Updated this week
- Evaluating majors LLMs on the Abstraction and Reasoning Corpus☆17Nov 9, 2023Updated 2 years ago
- Open deep learning compiler stack for cpu, gpu and specialized accelerators☆19Updated this week
- Transformers components but in Triton☆34May 9, 2025Updated 9 months ago
- Supporting code for the blog post on modular manifolds.☆117Sep 26, 2025Updated 5 months ago
- ☆19Dec 4, 2025Updated 2 months ago
- JAX/Flax implementation of the Hyena Hierarchy☆34Apr 27, 2023Updated 2 years ago
- Official pytorch implementation for our ICLR 2023 paper "Latent State Marginalization as a Low-cost Approach for Improving Exploration".☆24Feb 9, 2023Updated 3 years ago
- A curated list of awesome memory in reinforcement learning research materials☆24Sep 5, 2021Updated 4 years ago
- A benchmark to test and compare your pcg algorithm against each other☆22Sep 22, 2025Updated 5 months ago
- [ICLR2025 Spotlight🔥] Official Implementation of TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters☆588Feb 11, 2025Updated last year
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆187Jan 19, 2026Updated last month
- This repository contains the official code for Energy Transformer---an efficient Energy-based Transformer variant for graph classificatio…☆25Jan 28, 2024Updated 2 years ago
- A minimal home grid world environment to evaluate language understanding in interactive agents.☆24Sep 6, 2023Updated 2 years ago
- Our solution of the Kaggle Abstraction and Reasoning Challenge☆23May 30, 2020Updated 5 years ago
- Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy☆23Oct 28, 2024Updated last year
- train with kittens!☆63Oct 25, 2024Updated last year
- ☆35Nov 22, 2024Updated last year
- Collection of autoregressive model implementation☆85Updated this week
- [ICLR 2025] Official Pytorch Implementation of "Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN" by Pengxia…☆29Jul 24, 2025Updated 7 months ago
- ☆252Dec 2, 2024Updated last year
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters☆133Dec 3, 2024Updated last year
- NanoGPT-speedrunning for the poor T4 enjoyers☆73Apr 22, 2025Updated 10 months ago