DACUS1995 / pytorch-mmap-datasetLinks
A custom pytorch Dataset extension that provides a faster iteration and better RAM usage
☆44Updated last year
Alternatives and similar repositories for pytorch-mmap-dataset
Users that are interested in pytorch-mmap-dataset are comparing it to the libraries listed below
Sorting:
- PyTorch implementation of Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation☆29Updated 3 years ago
- Implementation of fused cosine similarity attention in the same style as Flash Attention☆215Updated 2 years ago
- The accompanying code for "Memory-efficient Transformers via Top-k Attention" (Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonatha…☆69Updated 3 years ago
- [ICLR 2022] Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention☆196Updated 2 years ago
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆76Updated last year
- Transformers w/o Attention, based fully on MLPs☆94Updated last year
- [NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"☆73Updated 2 years ago
- several types of attention modules written in PyTorch for learning purposes☆53Updated 10 months ago
- PyTorch, PyTorch Lightning framework for trying knowledge distillation in image classification problems☆32Updated last year
- [ICLR 2022] "As-ViT: Auto-scaling Vision Transformers without Training" by Wuyang Chen, Wei Huang, Xianzhi Du, Xiaodan Song, Zhangyang Wa…☆76Updated 3 years ago
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆48Updated 2 years ago
- Easily benchmark PyTorch model FLOPs, latency, throughput, allocated gpu memory and energy consumption☆105Updated last year
- A repository for DenseSSMs☆88Updated last year
- ☆95Updated 2 years ago
- A simple program to calculate and visualize the FLOPs and Parameters of Pytorch models, with handy CLI and easy-to-use Python API.☆129Updated 8 months ago
- Pytorch Implementation of the sparse attention from the paper: "Generating Long Sequences with Sparse Transformers"☆86Updated 3 weeks ago
- Root Mean Square Layer Normalization☆249Updated 2 years ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆39Updated 3 years ago
- code for the ddp tutorial☆32Updated 3 years ago
- Implementation of "Attention Is Off By One" by Evan Miller☆195Updated last year
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆60Updated last year
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆62Updated 3 years ago
- Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".☆44Updated 3 years ago
- ☆33Updated 2 months ago
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆61Updated 2 years ago
- Implementation of Multistream Transformers in Pytorch☆54Updated 4 years ago
- Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"☆369Updated last year
- Pytorch cyclic cosine decay learning rate scheduler☆49Updated 4 years ago
- [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling☆86Updated 2 years ago
- Implementation of Fast Transformer in Pytorch☆175Updated 3 years ago