fal-ai-community / nano-mdm
Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
☆48Updated last month
Alternatives and similar repositories for nano-mdm:
Users that are interested in nano-mdm are comparing it to the libraries listed below
- Focused on fast experimentation and simplicity☆71Updated 4 months ago
- research impl of Native Sparse Attention (2502.11089)☆53Updated 2 months ago
- ☆78Updated 8 months ago
- supporting pytorch FSDP for optimizers☆80Updated 4 months ago
- Official PyTorch Implementation for Paper "No More Adam: Learning Rate Scaling at Initialization is All You Need"☆51Updated 2 months ago
- ☆78Updated 9 months ago
- Repository for the Q-Filters method (https://arxiv.org/pdf/2503.02812)☆30Updated last month
- Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam☆75Updated 8 months ago
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training☆123Updated last year
- ☆33Updated 7 months ago
- WIP☆93Updated 8 months ago
- ☆27Updated 4 months ago
- ☆27Updated 11 months ago
- Code for the paper: "Fine-Tuning Discrete Diffusion Models with Policy Gradient Methods"☆19Updated last month
- ☆51Updated last year
- [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"☆95Updated 2 weeks ago
- Minimal Implementation of Visual Autoregressive Modelling (VAR)☆30Updated last month
- Triton Implementation of HyperAttention Algorithm☆47Updated last year
- Code accompanying the paper "Generalized Interpolating Discrete Diffusion"☆77Updated this week
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆105Updated 5 months ago
- ☆19Updated 3 weeks ago
- Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"☆25Updated 5 months ago
- Official Implementation for the paper "d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning"☆77Updated last week
- Fourier Position Embedding: Enhancing Attention’s Periodic Extension for Length Generalization☆19Updated 3 months ago
- Collection of autoregressive model implementation☆85Updated 2 months ago
- ☆53Updated last month
- ☆22Updated 10 months ago
- DPO, but faster 🚀☆40Updated 4 months ago
- ☆95Updated last year
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆55Updated 11 months ago