DACUS1995 / pytorch-mmap-dataset
A custom pytorch Dataset extension that provides a faster iteration and better RAM usage
☆43Updated last year
Alternatives and similar repositories for pytorch-mmap-dataset:
Users that are interested in pytorch-mmap-dataset are comparing it to the libraries listed below
- PyTorch implementation of Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation☆28Updated 3 years ago
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆60Updated last year
- several types of attention modules written in PyTorch for learning purposes☆48Updated 6 months ago
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆22Updated 9 months ago
- ☆30Updated 10 months ago
- A simple implementation of [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752)☆21Updated last year
- [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control variates, random features, and importance sampling☆82Updated 2 years ago
- PyTorch implementation of "From Sparse to Soft Mixtures of Experts"☆53Updated last year
- PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)☆71Updated last year
- 32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.☆47Updated last year
- Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr\"om Method (NeurIPS 2021)☆60Updated 2 years ago
- The accompanying code for "Memory-efficient Transformers via Top-k Attention" (Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonatha…☆66Updated 3 years ago
- Implementation of TableFormer, Robust Transformer Modeling for Table-Text Encoding, in Pytorch☆37Updated 3 years ago
- Distributed DataLoader For Pytorch Based On Ray☆24Updated 3 years ago
- code for the ddp tutorial☆32Updated 2 years ago
- A small framework mimics PyTorch using CuPy or NumPy☆27Updated 3 years ago
- [NeurIPS 2022 Spotlight] This is the official PyTorch implementation of "EcoFormer: Energy-Saving Attention with Linear Complexity"☆71Updated 2 years ago
- ☆16Updated 2 years ago
- ☆22Updated last year
- Sparse Attention with Linear Units☆17Updated 3 years ago
- A repository for DenseSSMs☆87Updated 11 months ago
- Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012☆49Updated 2 years ago
- Linear Attention Sequence Parallelism (LASP)☆81Updated 10 months ago
- Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".☆44Updated 3 years ago
- [KDD'22] Learned Token Pruning for Transformers☆96Updated 2 years ago
- [ICLR 2022] Official implementation of cosformer-attention in cosFormer: Rethinking Softmax in Attention☆190Updated 2 years ago
- HGRN2: Gated Linear RNNs with State Expansion☆54Updated 7 months ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆26Updated last year
- CUDA implementation of autoregressive linear attention, with all the latest research findings☆44Updated last year
- MambaFormer in-context learning experiments and implementation for https://arxiv.org/abs/2402.04248☆51Updated 9 months ago