DACUS1995 / pytorch-mmap-dataset
A custom pytorch Dataset extension that provides a faster iteration and better RAM usage
☆43Updated last year
Alternatives and similar repositories for pytorch-mmap-dataset:
Users that are interested in pytorch-mmap-dataset are comparing it to the libraries listed below
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆60Updated last year
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆47Updated last year
- # Unified Normalization (ACM MM'22) By Qiming Yang, Kai Zhang, Chaoxiang Lan, Zhi Yang, Zheyang Li, Wenming Tan, Jun Xiao, and Shiliang P…☆34Updated 2 years ago
- PyTorch implementation of Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation☆28Updated 3 years ago
- The accompanying code for "Memory-efficient Transformers via Top-k Attention" (Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonatha…☆67Updated 3 years ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆37Updated 3 years ago
- ☆30Updated 11 months ago
- Implementation of Infini-Transformer in Pytorch☆110Updated 3 months ago
- Memory Optimizations for Deep Learning (ICML 2023)☆64Updated last year
- Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding☆47Updated 6 months ago
- ☆48Updated last year
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆71Updated last year
- ☆22Updated last year
- A repository for DenseSSMs☆87Updated last year
- [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling☆84Updated 2 years ago
- Implementation of the Remixer Block from the Remixer paper, in Pytorch☆35Updated 3 years ago
- Context Manager to profile the forward and backward times of PyTorch's nn.Module☆83Updated last year
- Implementation of the Kalman Filtering Attention proposed in "Kalman Filtering Attention for User Behavior Modeling in CTR Prediction"☆57Updated last year
- ☆24Updated 2 years ago
- Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"☆52Updated 3 weeks ago
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆53Updated last year
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆25Updated 10 months ago
- several types of attention modules written in PyTorch for learning purposes☆50Updated 6 months ago
- differentiable top-k operator☆21Updated 3 months ago
- A simple implementation of [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752)☆21Updated last year
- Implementation of fused cosine similarity attention in the same style as Flash Attention☆213Updated 2 years ago
- CUDA implementation of autoregressive linear attention, with all the latest research findings☆44Updated last year
- A Tight-fisted Optimizer☆47Updated 2 years ago
- State-of-the-art data augmentation search algorithms in PyTorch☆47Updated last year
- Implementation of "compositional attention" from MILA, a multi-head attention variant that is reframed as a two-step attention process wi…☆50Updated 2 years ago