lucidrains / lion-pytorch
π¦ Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch
β2,002Updated 3 months ago
Related projects: β
- A concise but complete full-attention transformer with a set of promising experimental features from various papersβ4,573Updated last week
- Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.β2,206Updated 3 weeks ago
- A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.β2,206Updated this week
- Vector (and Scalar) Quantization, in Pytorchβ2,404Updated this week
- TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.β1,423Updated this week
- Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelsβ746Updated 2 months ago
- Foundation Architecture for (M)LLMsβ3,003Updated 5 months ago
- Structured state space sequence modelsβ2,361Updated 2 months ago
- Torchmetrics - Machine learning metrics for distributed, scalable PyTorch applications.β2,077Updated this week
- A method to increase the speed and lower the memory footprint of existing vision transformers.β934Updated 3 months ago
- maximal update parametrization (Β΅P)β1,334Updated 2 months ago
- EVA Series: Visual Representation Fantasies from BAAIβ2,209Updated last month
- torchview: visualize pytorch modelsβ793Updated 4 months ago
- Code release for ConvNeXt V2 modelβ1,467Updated last month
- View model summaries in PyTorch!β2,499Updated last week
- FFCV: Fast Forward Computer Vision (and other ML workloads!)β2,831Updated 3 months ago
- Scenic: A Jax Library for Computer Vision Research and Beyondβ3,248Updated 3 weeks ago
- minLoRA: a minimal PyTorch library that allows you to apply LoRA to any PyTorch model.β423Updated last year
- Tensors, for human consumptionβ1,096Updated 2 weeks ago
- SAM: Sharpness-Aware Minimization (PyTorch)β1,742Updated 6 months ago
- The official implementation of βSophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingββ931Updated 7 months ago
- [CVPR 2021] Official PyTorch implementation for Transformer Interpretability Beyond Attention Visualization, a novel method to visualize β¦β1,759Updated 7 months ago
- Implementation of 𦩠Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorchβ1,192Updated last year
- Schedule-Free Optimization in PyTorchβ1,800Updated last month
- π A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (iβ¦β7,687Updated this week
- Cramming the training of a (BERT-type) language model into limited compute.β1,284Updated 3 months ago
- PyTorch extensions for high performance and large scale training.β3,149Updated 2 weeks ago
- Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorchβ1,037Updated 9 months ago
- Neighborhood Attention Transformer, arxiv 2022 / CVPR 2023. Dilated Neighborhood Attention Transformer, arxiv 2022β1,040Updated 4 months ago
- Pytorch library for fast transformer implementationsβ1,621Updated last year