drisspg / transformer_nuggetsLinks
A place to store reusable transformer components of my own creation or found on the interwebs
☆60Updated last week
Alternatives and similar repositories for transformer_nuggets
Users that are interested in transformer_nuggets are comparing it to the libraries listed below
Sorting:
- Experiment of using Tangent to autodiff triton☆80Updated last year
- Triton Implementation of HyperAttention Algorithm☆48Updated last year
- Make triton easier☆48Updated last year
- ☆91Updated last year
- ☆21Updated 7 months ago
- Demo of the unit_scaling library, showing how a model can be easily adapted to train in FP8.☆45Updated last year
- Various transformers for FSDP research☆38Updated 2 years ago
- CUDA and Triton implementations of Flash Attention with SoftmaxN.☆73Updated last year
- Easily run PyTorch on multiple GPUs & machines☆47Updated 2 weeks ago
- ☆121Updated last year
- ☆20Updated 2 years ago
- Utilities for Training Very Large Models☆58Updated last year
- Jax like function transformation engine but micro, microjax☆33Updated last year
- Context Manager to profile the forward and backward times of PyTorch's nn.Module☆82Updated 2 years ago
- ☆21Updated last year
- Latent Diffusion Language Models☆68Updated 2 years ago
- This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog po…☆92Updated 2 years ago
- DPO, but faster 🚀☆45Updated 10 months ago
- An implementation of the Llama architecture, to instruct and delight☆21Updated 4 months ago
- A sample pattern for running CI tests on Modal☆18Updated 6 months ago
- Hacks for PyTorch☆19Updated 2 years ago
- some common Huggingface transformers in maximal update parametrization (µP)☆86Updated 3 years ago
- A library for unit scaling in PyTorch☆132Updated 3 months ago
- ☆46Updated last year
- Collection of autoregressive model implementation☆86Updated 6 months ago
- ☆52Updated last year
- Using FlexAttention to compute attention with different masking patterns☆47Updated last year
- ML/DL Math and Method notes☆64Updated last year
- Genalog is an open source, cross-platform python package allowing generation of synthetic document images with custom degradations and te…☆43Updated last year
- Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training☆132Updated last year