BorealisAI / efficient-vit-training
PyTorch code of "Training a Vision Transformer from scratch in less than 24 hours with 1 GPU" (HiTY workshop at Neurips 2022)
☆19Updated last year
Alternatives and similar repositories for efficient-vit-training:
Users that are interested in efficient-vit-training are comparing it to the libraries listed below
- Official PyTorch implementation of "LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging" (ICML'24)☆29Updated 5 months ago
- An efficient pytorch implementation of selective scan in one file, works with both cpu and gpu, with corresponding mathematical derivatio…☆78Updated 10 months ago
- Official Pytorch Implementation of Self-emerging Token Labeling☆32Updated 9 months ago
- The official implementation of LumiNet: The Bright Side of Perceptual Knowledge Distillation https://arxiv.org/abs/2310.03669☆20Updated 10 months ago
- Implementation of a modular, high-performance, and simplistic mamba for high-speed applications☆33Updated 2 months ago
- This repository contains the pytorch code for our ISBI 2024 paper "ConvLoRA and AdaBN Based Domain Adaptation via Self-Training".☆62Updated 3 months ago
- [ECCV 2024] Isomorphic Pruning for Vision Models☆61Updated 5 months ago
- Pytorch Implementation of the paper: "Learning to (Learn at Test Time): RNNs with Expressive Hidden States"☆24Updated last week
- Implementation of ViTaR: ViTAR: Vision Transformer with Any Resolution in PyTorch☆30Updated 2 months ago
- Simple Implementation of TinyGPTV in super simple Zeta lego blocks☆15Updated 2 months ago
- [EMNLP 2024] RWKV-CLIP: A Robust Vision-Language Representation Learner☆121Updated this week
- [ICCV2023] TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance☆77Updated 6 months ago
- The open source implementation of the model from "Scaling Vision Transformers to 22 Billion Parameters"☆27Updated this week
- Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"☆79Updated last week
- A big_vision inspired repo that implements a generic Auto-Encoder class capable in representation learning and generative modeling.☆34Updated 6 months ago
- Code Implementation of EfficientVMamba☆191Updated 9 months ago
- Timm model explorer☆36Updated 9 months ago
- Official Training and Inference Code of Amodal Expander, Proposed in Tracking Any Object Amodally☆14Updated 6 months ago
- Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Ze…☆90Updated this week
- Implementation of a Light Recurrent Unit in Pytorch☆47Updated 3 months ago
- Explorations into improving ViTArc with Slot Attention☆37Updated 3 months ago
- [ICML 2024] Official PyTorch implementation of "SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-paramete…☆86Updated 4 months ago
- Scaling RWKV-Like Architectures for Diffusion Models☆122Updated 9 months ago
- Stable Diffusion in TensorRT 8.5+☆14Updated last year
- The official project website of "KernelWarehouse: Rethinking the Design of Dynamic Convolution" (KW for short, accepted to ICML 2024)☆96Updated 7 months ago
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"☆96Updated 3 months ago
- Implementation of the "the first large-scale multimodal mixture of experts models." from the paper: "Multimodal Contrastive Learning with…☆25Updated 2 months ago
- OLA-VLM: Elevating Perception in Multimodal LLMs with Auxiliary Embedding Distillation, arXiv 2024☆45Updated last month
- The official implementation of the paper "Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation"☆19Updated last month
- A simpler Pytorch + Zeta Implementation of the paper: "SiMBA: Simplified Mamba-based Architecture for Vision and Multivariate Time series…☆27Updated 2 months ago