fal-ai-community / nano-mdmLinks
Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun
☆55Updated 4 months ago
Alternatives and similar repositories for nano-mdm
Users that are interested in nano-mdm are comparing it to the libraries listed below
Sorting:
- ☆83Updated 11 months ago
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆127Updated 11 months ago
- Code accompanying the paper "Generalized Interpolating Discrete Diffusion"☆97Updated last month
- Focused on fast experimentation and simplicity☆76Updated 7 months ago
- Implementation of a multimodal diffusion transformer in Pytorch☆102Updated last year
- research impl of Native Sparse Attention (2502.11089)☆60Updated 5 months ago
- Simple implementation of muP, based on Spectral Condition for Feature Learning. The implementation is SGD only, dont use it for Adam☆84Updated last year
- Implementation of 🥥 Coconut, Chain of Continuous Thought, in Pytorch☆178Updated last month
- Implementation of Infini-Transformer in Pytorch☆111Updated 7 months ago
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training☆130Updated last year
- supporting pytorch FSDP for optimizers☆84Updated 7 months ago
- ☆81Updated last year
- Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".☆131Updated last month
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆56Updated last year
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆149Updated last month
- Official repository of "LiNeS: Post-training Layer Scaling Prevents Forgetting and Enhances Model Merging"☆30Updated 9 months ago
- ☆104Updated 2 years ago
- [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"☆118Updated last month
- Official PyTorch Implementation for Paper "No More Adam: Learning Rate Scaling at Initialization is All You Need"☆52Updated 6 months ago
- Repository for the Q-Filters method (https://arxiv.org/pdf/2503.02812)☆34Updated 4 months ago
- A repository for research on medium sized language models.☆78Updated last year
- ☆60Updated 4 months ago
- WIP☆93Updated 11 months ago
- ☆34Updated 10 months ago
- [ICML 2025] Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction☆55Updated 2 months ago
- Collection of autoregressive model implementation☆86Updated 3 months ago
- Mixture of A Million Experts☆46Updated last year
- Esoteric Language Models☆89Updated last week
- Block Transformer: Global-to-Local Language Modeling for Fast Inference (NeurIPS 2024)☆160Updated 3 months ago
- DPO, but faster 🚀☆43Updated 8 months ago