facebookresearch / spdl
Scalable and Performant Data Loading
☆66Updated this week
Related projects ⓘ
Alternatives and complementary repositories for spdl
- PyTorch video decoding☆79Updated this week
- Accelerated First Order Parallel Associative Scan☆163Updated 3 months ago
- CUDA implementation of autoregressive linear attention, with all the latest research findings☆43Updated last year
- An implementation of the Llama architecture, to instruct and delight☆21Updated 3 months ago
- Just some miscellaneous utility functions / decorators / modules related to Pytorch and Accelerate to help speed up implementation of new…☆119Updated 3 months ago
- ☆77Updated 5 months ago
- Experiment of using Tangent to autodiff triton☆72Updated 9 months ago
- Implementation of the proposed Adam-atan2 from Google Deepmind in Pytorch☆94Updated 3 weeks ago
- A library for unit scaling in PyTorch☆105Updated 2 weeks ago
- ☆73Updated 4 months ago
- Multidimensional indexing for tensors☆113Updated last year
- Implementation of a Light Recurrent Unit in Pytorch☆46Updated last month
- Framework for writing deep learning training loops. Lightweight, and retaining full freedom to design as you see fits. It handles checkpo…☆103Updated 8 months ago
- ☆128Updated this week
- Exploration into the proposed "Self Reasoning Tokens" by Felipe Bonetto☆53Updated 6 months ago
- This repository contains the experimental PyTorch native float8 training UX☆211Updated 3 months ago
- My attempts at applying Soundstream design on learned tokenization of text and then applying hierarchical attention to text generation☆82Updated last month
- 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash…☆193Updated this week
- Implementation of Infini-Transformer in Pytorch☆104Updated last month
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training☆113Updated 7 months ago
- Implementation of GateLoop Transformer in Pytorch and Jax☆86Updated 5 months ago
- A simple library for scaling up JAX programs☆127Updated 2 weeks ago
- Normalized Transformer (nGPT)☆66Updated this week
- Implementation of a Transformer, but completely in Triton☆249Updated 2 years ago
- Griffin MQA + Hawk Linear RNN Hybrid☆85Updated 6 months ago
- Randomized Positional Encodings Boost Length Generalization of Transformers☆78Updated 8 months ago
- ☆48Updated this week
- WIP☆89Updated 3 months ago
- RWKV model implementation☆38Updated last year
- Automatically take good care of your preemptible TPUs☆32Updated last year