AnonymousAlethiometer / SGD_SaILinks
Official PyTorch Implementation for Paper "No More Adam: Learning Rate Scaling at Initialization is All You Need"
☆52Updated 6 months ago
Alternatives and similar repositories for SGD_SaI
Users that are interested in SGD_SaI are comparing it to the libraries listed below
Sorting:
- Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".☆131Updated last month
- A big_vision inspired repo that implements a generic Auto-Encoder class capable in representation learning and generative modeling.☆34Updated last year
- Tiny re-implementation of MDM in style of LLaDA and nano-gpt speedrun☆55Updated 4 months ago
- Implementation of the proposed MaskBit from Bytedance AI☆82Updated 8 months ago
- ☆39Updated last year
- Implementation of a multimodal diffusion transformer in Pytorch☆102Updated last year
- Implementation of TiTok, proposed by Bytedance in "An Image is Worth 32 Tokens for Reconstruction and Generation"☆176Updated last year
- Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind☆127Updated 11 months ago
- Recaption large (Web)Datasets with vllm and save the artifacts.☆52Updated 8 months ago
- Autoregressive Image Generation☆32Updated last month
- Official implementation of ECCV24 paper: POA☆24Updated 11 months ago
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆56Updated last year
- Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch☆112Updated 8 months ago
- imagetokenizer is a python package, helps you encoder visuals and generate visuals token ids from codebook, supports both image and video…☆35Updated last year
- ☆70Updated 8 months ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆98Updated 10 months ago
- ☆27Updated last year
- [ICLR 2025] Official PyTorch implementation of "Forgetting Transformer: Softmax Attention with a Forget Gate"☆118Updated last month
- Minimal Differentiable Image Reward Functions☆79Updated last week
- Attempt to make multiple residual streams from Bytedance's Hyper-Connections paper accessible to the public☆88Updated last month
- [ICLR 2025] Source code for paper "A Spark of Vision-Language Intelligence: 2-Dimensional Autoregressive Transformer for Efficient Finegr…☆76Updated 7 months ago
- ☆32Updated 9 months ago
- Explorations into adversarial losses on top of autoregressive loss for language modeling☆37Updated 5 months ago
- BFloat16 Fused Adam Operator for PyTorch☆15Updated 8 months ago
- TerDiT: Ternary Diffusion Models with Transformers☆71Updated last year
- DPO, but faster 🚀☆43Updated 7 months ago
- research impl of Native Sparse Attention (2502.11089)☆60Updated 5 months ago
- NeuMeta transforms neural networks by allowing a single model to adapt on the fly to different sizes, generating the right weights when n…☆43Updated 8 months ago
- Official codebase for Margin-aware Preference Optimization for Aligning Diffusion Models without Reference (MaPO).☆79Updated last year
- The official repo of continuous speculative decoding☆27Updated 4 months ago