thinking-machines-lab / manifoldsLinks
Supporting code for the blog post on modular manifolds.
β103Updated 2 months ago
Alternatives and similar repositories for manifolds
Users that are interested in manifolds are comparing it to the libraries listed below
Sorting:
- supporting pytorch FSDP for optimizersβ84Updated 11 months ago
- πSmall Batch Size Training for Language Modelsβ64Updated last month
- β41Updated last month
- β91Updated last year
- β68Updated last year
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrunβ57Updated 8 months ago
- DeMo: Decoupled Momentum Optimizationβ197Updated last year
- The simplest, fastest repository for training/finetuning medium-sized GPTs.β174Updated 5 months ago
- Code for the paper "Function-Space Learning Rates"β23Updated 5 months ago
- H-Net Dynamic Hierarchical Architectureβ80Updated 2 months ago
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT trainingβ132Updated last year
- research impl of Native Sparse Attention (2502.11089)β63Updated 9 months ago
- β30Updated last year
- Accelerated First Order Parallel Associative Scanβ192Updated last year
- WIPβ93Updated last year
- β119Updated 5 months ago
- β61Updated last year
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmindβ131Updated last month
- Code for NeurIPS 2024 Spotlight: "Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations"β85Updated last year
- Focused on fast experimentation and simplicityβ75Updated 11 months ago
- β89Updated last year
- Normalized Transformer (nGPT)β194Updated last year
- Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clustersβ130Updated last year
- Landing repository for the paper "Softpick: No Attention Sink, No Massive Activations with Rectified Softmax"β85Updated 2 months ago
- β48Updated last month
- Mixture of A Million Expertsβ50Updated last year
- Explorations into the recently proposed Taylor Series Linear Attentionβ100Updated last year
- [ICLR 2025 & COLM 2025] Official PyTorch implementation of the Forgetting Transformer and Adaptive Computation Pruningβ134Updated 3 weeks ago
- Tiled Flash Linear Attention library for fast and efficient mLSTM Kernels.β74Updated last week
- Flash-Muon: An Efficient Implementation of Muon Optimizerβ212Updated 5 months ago