learning-at-home / lean_transformer
Memory-efficient transformer. Work in progress.
☆19Updated 2 years ago
Alternatives and similar repositories for lean_transformer:
Users that are interested in lean_transformer are comparing it to the libraries listed below
- ☆29Updated 2 years ago
- Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing☆48Updated 3 years ago
- DiCE: The Infinitely Differentiable Monte-Carlo Estimator☆31Updated last year
- Implementation of some personal helper functions for Einops, my most favorite tensor manipulation library ❤️☆54Updated 2 years ago
- ☆27Updated 4 years ago
- A case study of efficient training of large language models using commodity hardware.☆69Updated 2 years ago
- An open source implementation of CLIP.☆32Updated 2 years ago
- High performance pytorch modules☆18Updated 2 years ago
- Implementation of Kronecker Attention in Pytorch☆18Updated 4 years ago
- An attempt to merge ESBN with Transformers, to endow Transformers with the ability to emergently bind symbols☆15Updated 3 years ago
- Another attempt at a long-context / efficient transformer by me☆37Updated 3 years ago
- ☆31Updated last week
- A place to store reusable transformer components of my own creation or found on the interwebs☆49Updated this week
- A framework for implementing equivariant DL☆10Updated 3 years ago
- Collaborative inference of latent diffusion via hivemind☆12Updated last year
- Local Attention - Flax module for Jax☆20Updated 3 years ago
- ☆21Updated 2 years ago
- Hacks for PyTorch☆19Updated 2 years ago
- Official code for "Distributed Deep Learning in Open Collaborations" (NeurIPS 2021)☆116Updated 3 years ago
- "Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts" (NeurIPS 2020), original PyTorch implemen…☆54Updated 4 years ago
- A GPT, made only of MLPs, in Jax☆57Updated 3 years ago
- Training a model similar to OpenAI DALL-E with volunteers from all over the Internet using hivemind and dalle-pytorch (NeurIPS 2021 demo)☆26Updated last year
- ☆15Updated 3 years ago
- Contains my experiments with the `big_vision` repo to train ViTs on ImageNet-1k.☆22Updated 2 years ago
- A lightweight wrapper for PyTorch that provides a simple declarative API for context switching between devices, distributed modes, mixed-…☆67Updated last year
- PyTorch implementation of GLOM☆22Updated 3 years ago
- Latest Weight Averaging (NeurIPS HITY 2022)☆30Updated last year
- Triton Implementation of HyperAttention Algorithm☆47Updated last year
- My explorations into editing the knowledge and memories of an attention network☆34Updated 2 years ago
- Load any clip model with a standardized interface☆21Updated 11 months ago