zqOuO / GWTLinks
☆13Updated 4 months ago
Alternatives and similar repositories for GWT
Users that are interested in GWT are comparing it to the libraries listed below
Sorting:
- ☆32Updated 2 months ago
- ☆12Updated 3 months ago
- This repository contains code for the MicroAdam paper.☆19Updated 5 months ago
- ☆52Updated last year
- ☆79Updated 9 months ago
- ☆9Updated last year
- Code for "RSQ: Learning from Important Tokens Leads to Better Quantized LLMs"☆17Updated 2 weeks ago
- This repo is based on https://github.com/jiaweizzhao/GaLore☆28Updated 8 months ago
- Work in progress.☆67Updated this week
- An extention to the GaLore paper, to perform Natural Gradient Descent in low rank subspace☆17Updated 7 months ago
- Official implementation of "The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs"☆32Updated last month
- Unofficial Implementation of Selective Attention Transformer☆16Updated 7 months ago
- ☆19Updated 2 months ago
- Flash-Muon: An Efficient Implementation of Muon Optimizer☆121Updated last week
- A fusion of a linear layer and a cross entropy loss, written for pytorch in triton.☆67Updated 10 months ago
- ☆46Updated last year
- Here we will test various linear attention designs.☆58Updated last year
- ☆15Updated last year
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun☆52Updated 2 months ago
- Why Do We Need Weight Decay in Modern Deep Learning? [NeurIPS 2024]☆66Updated 8 months ago
- ☆53Updated 8 months ago
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- Latest Weight Averaging (NeurIPS HITY 2022)☆30Updated last year
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆47Updated last month
- Efficient PScan implementation in PyTorch☆16Updated last year
- Implementation of GateLoop Transformer in Pytorch and Jax☆88Updated 11 months ago
- Official repository for the paper "Approximating Two-Layer Feedforward Networks for Efficient Transformers"☆37Updated last year
- ☆25Updated 6 months ago
- [NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Se…☆64Updated last year
- The official repository for SkyLadder: Better and Faster Pretraining via Context Window Scheduling☆32Updated 2 months ago