lucidrains / lion-pytorchLinks
π¦ Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch
β2,132Updated 6 months ago
Alternatives and similar repositories for lion-pytorch
Users that are interested in lion-pytorch are comparing it to the libraries listed below
Sorting:
- Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelsβ791Updated 10 months ago
- Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.β2,900Updated last week
- torchview: visualize pytorch modelsβ937Updated last week
- Foundation Architecture for (M)LLMsβ3,076Updated last year
- Tensors, for human consumptionβ1,252Updated last week
- TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.β1,602Updated this week
- A simple way to keep track of an Exponential Moving Average (EMA) version of your Pytorch modelβ587Updated 5 months ago
- FFCV: Fast Forward Computer Vision (and other ML workloads!)β2,930Updated 11 months ago
- Scenic: A Jax Library for Computer Vision Research and Beyondβ3,542Updated this week
- Structured state space sequence modelsβ2,629Updated 10 months ago
- Implementation of Rotary Embeddings, from the Roformer paper, in Pytorchβ681Updated 6 months ago
- Machine learning metrics for distributed, scalable PyTorch applications.β2,279Updated this week
- Schedule-Free Optimization in PyTorchβ2,161Updated last week
- torch-optimizer -- collection of optimizers for Pytorchβ3,113Updated last year
- SAM: Sharpness-Aware Minimization (PyTorch)β1,874Updated last year
- A method to increase the speed and lower the memory footprint of existing vision transformers.β1,054Updated 11 months ago
- D-Adaptation for SGD, Adam and AdaGradβ518Updated 4 months ago
- A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.β2,615Updated this week
- The official implementation of βSophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingββ958Updated last year
- EVA Series: Visual Representation Fantasies from BAAIβ2,496Updated 9 months ago
- maximal update parametrization (Β΅P)β1,526Updated 10 months ago
- Hiera: A fast, powerful, and simple hierarchical vision transformer.β985Updated last year
- VISSL is FAIR's library of extensible, modular and scalable components for SOTA Self-Supervised Learning with images.β3,284Updated last year
- [CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize β¦β1,887Updated last year
- A concise but complete full-attention transformer with a set of promising experimental features from various papersβ5,328Updated 2 weeks ago
- View model summaries in PyTorch!β2,792Updated this week
- Vector (and Scalar) Quantization, in Pytorchβ3,269Updated 3 weeks ago
- Cramming the training of a (BERT-type) language model into limited compute.β1,332Updated 11 months ago
- Official DeiT repositoryβ4,205Updated last year
- β775Updated last month